Maybe some use cases will help to clarify things.
-- case 1)
<unit id='1'>
<segment translate='no'>
<source>t1 <sm id='1' translate='yes'/></source>
</segment>
<segment translate='no'>
<source>t2 <em startRef='1'/>t3</source>
</segment>
-> t2 is translatable
-> t3 is not translatable
-- case 2)
<unit id='1' translate='no'>
<segment translate='no'>
<source>t1 <sm id='1' translate='yes'/></source>
</segment>
<segment>
<source>t2 <em startRef='1'/>t3</source>
</segment>
-> t2 is translatable
-> t3 is not translatable
-- case 3)
<unit id='1' translate='yes'>
<segment translate='no'>
<source>t1 <sm id='1' translate='yes'/></source>
</segment>
<segment>
<source>t2 <em startRef='1'/>t3</source>
</segment>
-> t2 is translatable
-> t3 is translatable or not???
-- case 4)
<unit id='1' translate='no'>
<segment translate='yes'>
<source>t1 <sm id='1' translate='no'/>t2 <sm id='2' translate='yes'/></source>
</segment>
<segment>
<source>t3 <em startRef='2'/>t4 <em startRef='1'/>t5</source>
</segment>
-> t3 is translatable
-> t4 is not translatable
-> t5 is translatable or not???
-- case 5)
<unit id='1' translate='no'>
<segment translate='yes'>
<source>t1 <sm id='1' translate='no'/>t2 <sm id='2' translate='yes'/></source>
</segment>
<segment translate='yes'>
<source>t3 <em startRef='2'/>t4 </source>
</segment>
<segment>
<source><em startRef='1'/>t5</source>
</segment>
-> t3 is translatable
-> t4 is not translatable (the translate='yes' in that <segment> is non-applicable)
-> t5 is translatable or not???
-ys