Re: Fw: [xliff] Segmentation Modifications

Hi. Moving this to the XLIFF list for discussion.

We do not really use the dir attribute
within IBM as described. I'd like to put this out on the XLIFF mailing
list for others to comment.

Thanks.

From:      
 "Aharon (Vladimir)
Lanin" <>

To:      
 Steven R Loomis/Cupertino/IBM@IBMUS

Cc:      
 Richard Ishida <>,
Helena S Chapman/San Jose/IBM@IBMUS, Michael Ow/Southbury/IBM@IBMUS

Date:      
 12/15/2013 05:50 AM

Subject:    
   Re: Fw: [xliff]
Segmentation Modifications

Caveat: I know very little about XLIFF outside of what
I found in the Wikipedia article and some parts of the spec before writing
this email.

I think that the place to start is to take a step backwards.
The basic reason that we have directionality in text is that different
scripts have different directionalities. To put it another way, 99% of
the time, the directionality of a piece of text stems from the script in
which it is written. Thus, for example, Hebrew text is RTL and Latin text
is LTR. Now, when I embed a Latin phrase (e.g. a movie name or a street
address) in some Hebrew text, or vice versa, it is important to indicate
the directionality of the embedded text, since otherwise the embedded text
has a good chance of not being displayed as intended (e.g. "19 Main
Street, Oakland" when displayed RTL comes out looking like "Main
Street, Oakland 19" - which makes as little sense in a Hebrew document
as it does in an English one). And thus, we have the directional formatting
characters of Unicode and the dir attribute in HTML and XML. My point,
however, is that in 99% of the cases, the reason a direction
switch is needed is that a script switch has taken place. The way
to indicate a script (in XLIFF as in HTML, XML, etc.), whether implicitly
or explicitly, is with the lang attribute. It is true that Unicode
does not have formatting characters to indicate language or script, and
the lang attribute of HTML/XML is used quite rarely, but that is only because
in most cases one does not need to know the language of a piece of text
in order to display it as intended. Without knowing the directionality,
however, one will often display the text garbled. Thus, directionality
gets indicated a lot more often than the language - but the underlying
cause of directionality changes is still a change in script, even though
most of the time it does not get indicated explicitly.

Now, in XLIFF, the only elements allowed to
have xml:lang are <source> and <target>. Furthermore:

"When a <source> element is a child of <segment>
or <ignorable> and the OPTIONAL xml:lang attribute is present, its
value MUST be equal to the value of the srcLang attribute of the enclosing
<xliff> element."

"When a <target> element is a child of <segment>
or <ignorable> and the OPTIONAL xml:lang attribute is present, its
value MUST be equal to the value of the trgLang attribute of the enclosing
<xliff> element."

Thus, effectively, the entire XLIFF document has just
one source language and just one target language.

In contrast, XLIFF restricts the value of the dir attribute
on neither <source> nor <target>. Furthermore, the dir attribute
is also allowed on <data>, <pc> and <sc>.

I do not understand this.

If the <source> of segment 1 and the source of segment
2 are in the same language and script, why would one need to be LTR while
the other needs to be RTL?

The only ghost of a reason I can think of is that the
LTR segment is actually a mathematical formula or something like it, since
in several RTL language mathematics are nevertheless written LTR, but I
have trouble believing that this is really the reason for this anomaly
in XLIFF (or that there isn't a better way of indicating that a segment
is actually a mathematical _expression_ than by using the dir attribute).

In any case, assuming that there is a good reason, if
the dir attribute is available on <pc>, and <pc dir="ltr|rtl">
basically has the same meaning as <span dir="ltr|rtl">
in HTML, then why not indicate the directionality of segments that got
combined by adding a <pc> around the text from each one?

What is the purpose of the dir attribute on <data>?

Aharon

On Fri, Dec 13, 2013 at 9:18 PM, Steven R Loomis <>
wrote:

Richard, Aharon,

 FYI The  XLIFF  translation standard is trying to come
up with recommendations around their use of directional control chars vs.
markup.  Would either of you be able to provide some input? I can
see if they can put together some of their questions.

Thanks,

Steven

----- Forwarded by Steven R Loomis/Cupertino/IBM on 13/12/2013 11:16 -----

From: "Dr. David Filip"
<>

To: Yves Savourel <>

Cc: Steven R Loomis/Cupertino/IBM@IBMUS,
""
<>

Date: 13/12/2013 06:36

Subject: Re: [xliff] Segmentation
Modifications

Stephen, Yves, Fredrik, all,

I was looking up the bidi algorithm UAX#9, and I am not sure if we should
be using the explicit directionality control characters. The UAX#9 itself
quotes UTR#20, http://www.w3.org/TR/unicode-xml/ which
discourages the use of control characters in markup environment.

I know that XLIFF 1.2 did not have anything else, but why not have a full
markup solution this time round..

I wonder if we should rather use directionality annotations based on markers,
or dedicated directionality elements.

Another related issue is that both Unicode 6.3 and HTML 5 now allow for
heuristic determination of the directionality by the first strong character,
and there might be cases where this cannot be resolved into an explicit
directionality becuase of varaibales..

So whether we use control characters or if we go for marker based directionality
markup or even for dedicated directionality elements similar to HTML bdi,
we should have a value equivalent to FSI and bdi="auto"

Rgds

dF

Dr. David Filip

=======================

LRC | CNGL | LT-Web | CSIS

University of Limerick, Ireland

telephone: +353-6120-2781

cellphone: +353-86-0222-158 

facsimile: +353-6120-2734

http://www.cngl.ie/profile/?i=452

mailto: 

On Thu, Dec 12, 2013 at 6:41 PM, Yves Savourel <>
wrote: 

Thanks Steven,

Exactly the type of feedback
I was looking for. 

 

So we should do RLI+PDI
and LRI+PDI instead of RLE+PDF and LRE+PDF  I suppose?

 

-ys

 

From: Steven R Loomis [mailto:]

Sent: Thursday, December 12, 2013 10:55 AM

To: Yves Savourel

Cc: 

Subject: RE: [xliff] Segmentation Modifications 

  

Jumping in here..

 Please note that Unicode 6.3 adds directional isolate characters,
which could be useful for joining segments.

See:  http://www.unicode.org/reports/tr9/#Directional_Formatting_Characters 

Directional isolate characters were introduced in Unicode 6.3 after it
became apparent that directional embeddings usually have too strong an
effect on their surroundings and are thus unnecessarily difficult to use.
The new characters were introduced instead of changing the behavior of
the existing ones because doing so might have had an undesirable effect
on those existing documents that do rely on the old behavior. Nevertheless,
the use of the directional isolates instead of embeddings is encouraged
in new documents – once target platforms are known to support them.

-s

Yves
Savourel ---12/12/2013 05:50:43---For reference, the bidi text I’m talking
about is this one: [[

From: Yves Savourel <>

To: <>

Date: 12/12/2013 05:50

Subject: RE: [xliff] Segmentation Modifications

Sent by: <>

For reference, the bidi text I’m talking about is this one:

[[

If the dir attributes of the <source> or <target> elements
differ: The content of the <source> or <target> elements set
to a

different directionality than the directionality for the <source>
or <target> elements of the joined segment MUST be enclosed

between Unicode bi-directional control characters reflecting their original
directionality (U+202A and U+202C for left-to-right

spans, and U+202B and U+202C for right-to-left spans).

]]

From the attached file in this post:

https://lists.oasis-open.org/archives/xliff/201311/msg00176.html

The question is basically: are those Unicode control characters the one
to use for this mapping?

I based the text on this article:

http://www.w3.org/International/questions/qa-bidi-controls

Thanks,

-yves

From: Yves Savourel [mailto:]

Sent: Thursday, December 12, 2013 6:04 AM

To: ''

Subject: RE: [xliff] Segmentation Modifications

Hi David,

I can do the change, that will free you time for other ones.

Did you double check the bidi mapping?

I’m not expert on bidi, so it’d be good to have more than my input on
that part.

Cheers,

-yves

From: Dr. David Filip [mailto:]

Sent: Thursday, December 12, 2013 5:48 AM

To: Yves Savourel

Cc: 

Subject: Re: [xliff] Segmentation Modifications

Yves, all I did not hear any dissent on that

As far as i checked this, your proposal is equivalent to what was there
for csprd02 with two small exceptions that add to clarity:

1) You use an explicit bidi provision, so that people do not need to research
the Unicode BiDi algorithm for merging segments with

different dir

2) You also proposed to have an option to downgrade state on split segments,
which makes sense to me

Otherwise it is is just reorganizing the PRs by the perfomred type of modification,
which seems fine and I do not have a preference

regarding the presentation of the provisions.

@Yves, Do you want to implement this proposal in the spec or should I?

Please let me know

Thanks

dF

Dr. David Filip

=======================

LRC | CNGL | LT-Web | CSIS

University of Limerick, Ireland

telephone: +353-6120-2781

cellphone: +353-86-0222-158 

facsimile: +353-6120-2734

http://www.cngl.ie/profile/?i=452

mailto: 

On Sat, Nov 30, 2013 at 1:56 PM, Yves Savourel <>
wrote:

Hi all,

 

As mentioned here: https://lists.oasis-open.org/archives/xliff/201311/msg00138.html,
I've been trying to implement segmentation

modification for XLIFF 2.0 for a while now and I have a few comments.

 

For reference, the cs02 section for this is here:

http://docs.oasis-open.org/xliff/xliff-core/v2.0/csprd02/xliff-core-v2.0-csprd02.html#d0e9317

 

 

--- The section (starting with its new title) keeps talking about "segmentation
modification" and "resegmentation". Could we just

talk about segmentation modification everywhere? The two things are the
same thing.

 

 

--- That section has many constraints and processing requirements.

It was quite difficult to follow when I tried to implement it.

 

For example: (take a deep breath) "Modifiers MUST copy all attributes
including values, except for the id and order attributes, from

their original instances on or within the original <segment> element
onto both instances on and within the resulting two <segment>

or <ignorable> elements, except for attributes that do not have valid
instances on the eventually resulting <ignorable> element."

 

To make a long story short and get to the point, I think that section should
be re-worded to be simpler, organized by action (split

or join), and completed with a few things (some subState PRs, explicit
directionality conversion, etc.)

 

The proposed modified text is in the attached document.

 

I believe it covers what is needed, but it's a complex set of PRs and it
should be carefully checked by all. For example I'd like a

confirmation on the Unicode control characters used for the directionality
conversion.

 

Thanks,

-yves

 

 

---------------------------------------------------------------------

To unsubscribe from this mail list, you must leave the OASIS TC that

generates this mail.  Follow this link to all your TCs in OASIS at:

https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

---------------------------------------------------------------------

To unsubscribe from this mail list, you must leave the OASIS TC that 

generates this mail.  Follow this link to all your TCs in OASIS at:

https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php