Issue 12007 : Indirect reference/keyref proposal

Attached
is the current draft of the proposal for DITA Issue # 12007 : Indirect
referencing based on key references and key definitions to increase reusability
of content.

This was
originally proposed feature #40 (11040).

The current draft
is based on an initial draft by Michael Priestly and is the work of an ad hoc
subcommittee convened by Michael Priestly that includes: Michael Priestley,
Jeff Ogden, Erik Hennum, Deborah Pickett; Ian Larner; Michael Gannon, and Eliot Kimber.

This proposal is
not yet ready for discussion by the full DITA TC, but the ad hoc subcommittee
would welcome comments and suggestions on the current approach as outlined in
the draft. Send your comments to this list () or to  or to .

  -Jeff Ogden

Title: DITA Proposed Feature # 40

DITA Issue # 12007 (draft
version 2.1, 29 October 2007)

Indirect referencing based on key references and key definitions to increase
reusability of content.

This was originally proposed feature #40 (11040).

The current draft is based on an initial draft by Michael Priestly and is
the work of an ad hoc subcommittee convened by Michael Priestly that includes:
Michael Priestley, Jeff Ogden, Erik Hennum, Deborah Pickett; Ian Larner;
Michael Gannon, and Eliot Kimber.

This proposal is not yet ready for discussion by the full DITA TC, but the
ad hoc subcommittee would welcome comments and suggestions on the current
approach as outlined here. Send comments to the DITA TC list or to  or to .

Longer description

Whenever a topic has a reference to other content, it makes the topic less
reusable because of the dependency on the target being still available and
still relevant. This proposal introduces a simple key-based redirection scheme
that leverages existing attributes and map architectures to provide support for
redirectable conrefs, topicrefs, xrefs, and other reference elements. It also
provides a simplified architecture for managing variable or volatile content
such as product names, which need to be easily swapped out when a topic is
reused in new contexts.

Key names are defined using  a topicref or
topocref specialization that uses the new keys attribute. A keys
definition defines one or more key names and the value to which the keys
resolve. Keys resolve to the URI given as the href value or to the contents of
a linktext element contained within a topicref or topicref specialization. 

Key definitions are used to resolve references
when a reference attribute (href, conref, …) on an element in a topic or a map
contains a key reference. A key reference consists of the string “ditakey:”
followed by a key name, optionally followed by a “/” character and an id of a
sub-topic element or by a “#” character and an id of a map element. If a key
reference cannot be resolved, the reference is resolved using an optional
fallback reference attribute on the original reference element.

Scope

Major

Use Case 1: Redirect link or xref

 
Author
     1 creates a map that associates keys with each topic, for example
     <topicref keys="a" href=""a.dita"/>

 
Author
     1 creates topic c.dita that contains a related link to a.dita - but uses
     the keyref attribute: <link href=""ditakey:a""
     fbref="a.dita"/>

 
Author
     2 reuses c.dita, but wants to redirect the link, so applies a different
     map with <topicref keys="a" href=""a1.dita"/>."
     The link in c.dita now resolves to a1.dita when author 2 builds it (it
     continues to resolve to a.dita when author 1 builds it)

 
Author
     3 also reuses c.dita, but wants the link to point to an external resource,
     so creates an external-pointing topicref to resolve the key: <topicref
     keys="a" href=""http://www.a...""
     scope="external"><topicmeta><linktext>This links
     to A2</linktext><desc>Because it does.</desc></topicmeta></topicref>.
     The link in c.dita now resolves to an external URI when author 3 builds it
     (without affecting how it resolves for the other two reusers).

 
Author
     4 wants to get rid of the link, so creates an explicitly empty topicref to
     get rid of it: <topicref keys="a"/>. This gets rid of the
     link for author 4 without affecting the other reusers.

 
Author
     5 wants to turn the link into just plain text (not hypertext):
     <topicref keys="a"><topicmeta><linktext>This
     is just text.</linktext></topicmeta></topicref> - for
     example a citation of a print-only magazine article...

Note: Should have a warning when an
inline xref is removed.

Use case 2: Redirect conref

 
Author
     1 creates a map that associates a key with a topic that contains reusable
     elements, for example <topicref keys="reuse" href=""prodA/reuse.dita"/>

 
Author
     1 uses the key instead of the full href whenever creating conrefs - for
     example <p conref="ditakey:reuse/para1"/>

 
Author
     2 wants to reuse author 1's content, but swap in a different set of
     reusable content. So Author 2 associates the key "reuse" with a
     different topic: <topicref keys="reuse"
     href=""prodB/mytopic.dita"/>." So now <p
     conref="ditakey:reuse/para1"/> will resolve to a paragraph
     with the id “para1” in prodB/mytopic.dita when author 2 builds the
     content, while continuing to resolve to athe para with the id “para1” in
     prodA/reuse.dita for author 1.

Note: The reusing author must create a
parallel set of elements and ids in the replacement topic - this proposal does
not remap the element ids within the topic, only the pointer to the topic
container.

Use case 3: Create links from keywords/terms/etc.

 
Author
     1 creates a map that contains glossary entries, and associates keys for
     each entry: <topicref keys="myterm"
     href=""myterm.dita"/>

 
Author
     1 then uses the keys to create links to the appropriate glossary entry
     from occurrences of terms in content: <term
     keyref="ditakey:myterm">my term</term>.

Use case 4: Swap out variable content

 
Author
     1 creates a map for key words and phrases that tend to change, such as ui
     labels and product names. The topicrefs do not in this case contain any
     actual hrefs, just the text that should be used: <topicref
     keys="prodname"><topicmeta><linktext>My
     Product</linktext></topicmeta></topicref>

 
Author
     1 then uses the keys to draw text into empty keywords: <keyword
     keyref="ditakey:prodname"/>

 
Author
     2 reuses the content but wants to use a different product name, so
     associates prodname with a different string: <topicref
     keys="prodname"><topicmeta><linktext>Another
     Product</linktext></topicmeta></topicref>. The keyword
     now resolves to "Another Product" for author 2, while continuing
     to resolve to "My Product" for author 1.

Note: Should generate a warning
message when a key reference on an empty element cannot be resolved, resulting
in the element effectively being removed.

Use case 5: Splitting/combining targets

 
Author
     1 creates a map in which most branches have the same structure: intro,
     example, reference. But two branches have only very little content in
     them, because the product support is only minimal. But in anticipation of
     future elaboration, author 1 assigns 3 keys to the container under which
     more topics are expected in the future: <topicref
     keys="blatoverview blatintro blatexample blatreference"
     href=""blatoverview.dita"/>

 
Author
     2 references ditakey:blatexample, and in the future when Author 1 moves
     blatexample into a separate topic, author 2's link remains appropriate and
     valid and does not need to be reworked.

 
Author
     3 is reusing a bunch of author 1's content, but in a context where blats
     are not available, and are instead replaced by foobars. So author 3 simply
     adds the blat keys to their own foobar topicref: <topicref
     keys="blatoverview blatintro blatexample blatreference foobar"
     href=""foobar.dita"/>

Technical Requirements

Add keys and usage attributes to topicref element in map.

Add keyref attribute to selected elements in topic and map [need a complete
list of elements].

Remove keyref attribute from and add fbref (fallback reference) attribute to
selected elements in topic and map [need a complete list of elements]. 

Document  the keys attribute and key name syntax: The keys attribute value
consists of one or more space separated key names. Key names consist of
characters that are legal in a URI and which cannot contain the “/”, “#”, or
space  characters. The use of periods in key names is discouraged  to allow the
use of periods to establish a hierarchal key name space in the future.

Document key reference syntax:
the string “ditakey:” followed by a key name optionally followed by / and topic
element id or # and mapid; for example,

 
ditakey:keyname

 
ditakey:keyname/elementid

 
ditakey:keyname#mapid

Key references can be added to or used from any of the existing
reference attributes (href, conref, …) in topics or maps [need a complete list
of elements]. When an elementid or a mapid is present as part of a key
reference, the id is combined with the resolved key definition value to make
the complete reference.

Document
fbref value syntax and semantics: a non-keyref reference value that when
present will be used in place of the primary reference attribute when the
primary reference attribute includes a key reference that cannot be resolved.

Document
usage attribute syntax and semantics: a list of one or more space separated
tokens that indicate the role or roles that the topicref or topicref
specialization is playing.  Like class and domains, the usage attribute value
will usually be defined in the DTD or schema rather than in the document
instance. Standard usage tokens are “content”, “relationships”, and “properties”.
Additional new usage tokens may be added.  The three standard tokens  will be
included as the default value for @usage, but this could be changed for a
specific topicref specialization.  To avoid name conflicts it is suggested that
user’s adopt the same prefixing scheme for user defined usage tokens as is
currently suggested for chunk token names. [Need more detail on the semantics
of each of the three standard usage tokens.]

Document required behavior in
language and architectural specs, as follows:

Keys are defined in maps and not in topics. Keys resolve to
topics, to collections of topics (ditabase), to maps, to referencable portions
of maps, to non-DITA documents, to external URIs, or to text strings. Keys do
not resolve to sub-topic elements.

Key definitions are not scoped by the map in which they occur or
by nesting within a map.
Key definitions and key references are processed in
the order in which they are encountered. An earlier key definition is
overridden by a later key definition. Key references included in conref
attributes are resolved before the conref is processed.

Key references are resolved as
follows:

 

 
For a match
     to a key definition with @href:

     Element with key reference is turned into link to the referenced target;
     if element is empty, text of element is pulled from topicref.

 
For a match
     to a key definition with title but no @href:

     Element with key reference is not turned into link. If element is empty,
     text of element is pulled from topicref's linktext.

 
For a match
     to a key definition with neither title nor @href:

     Element with key reference is not turned into link. If element is empty,
     element is left empty.

 
No match to a
     key definition:

     Element with key reference is not turned into link. If there is a fbref
     attribute on the referencing element, it is used as the reference.
     Otherwise, if the referencing element is empty, it is left empty and a
     warning emitted by the key reference processor.

The common attributes between a key reference element and a key
definition element other than the keys and id attributes are combined in a
fashion that is similar to the combining that is defined for attributes on
source and target conref elements today including the special processing for
the xml:lang, dir, and translate attributes.

When they are combined, attributes on a key reference element take
precedence over attributes on a key definition element. For a chain of key
reference elements the priority for combining attributes is:

1.       key
reference element

2.       1st keys definition element

3.       2nd keys definition element

4.       . . .

5.       last keys definition element

The combined attributes may cascade from one map to another or
from a map to a topic, but this is controlled by existing rules for cascading
which do not change with the use of key references.

Costs

Toolkit will need to include preprocessing pass to turn key references into
file/topic references for HTML or other output. Usability requirements on
editors to support key-based references, for example so that conrefs can
continue being resolved in the editor, even though the conref attribute on its
own is no longer enough to locate the content (the conref must be interpreted
in the context of a map file that associates the target key with a particular
topic). This puts a further premium on supporting map-based authoring, i.e.,
authoring a topic in the context of a map, rather than as isolated chunks.

Benefits

A major increase in reusability of content. Removes need for
file-system-centric workarounds such as conref substitution. Opportunity for
improved compatibility between different content management systems, and
between CMSs and other systems such as file systems, dynamic web publishing
systems, databases, etc. - each of which could use their own key scheme and
just externalize it as a map file for the keyed content, as well as remapping
keys to the preferred linking or retrieval mechanism at runtime.

Time Required

5 one hour meetings to agree on requirements and solution (hoping, three
down, two to go)

2 days to document

? days to implement changes to DTDs and schemas

? days to implement changes to DITA Open Toolkit and other output processors

Issues

 
We have pretty much agreed that it
     is not necessary to support direct remapping below the topic element
     level. We want to confirm  with others that this limitation is acceptable.
     If it is, it buys us a lot in terms of compartmentalizing the problem and
     maintaining architectural consistency (maps only reference topics, stuff
     below the topic level is handled within topics not by maps).

 
We
     aren’t particularly happy with the “ditakey:keyname” syntax. Can we come
     up with something better? The creation of new
     top level URI schemes such as “ditakey” is being discouraged. Other
     options that have been considered are to use a special character such as
     dollar-sign ($) to flag the keyref value or to enclose the key name in
     [square] or {curly} brackets. While there are drawbacks to using a scheme
     such as ditakey:, there are also issues of needing to make the reference
     value something that could be a valid URI. A concern is that to make a
     keyref a valid URI that the reference would need to start with either a
     #-sign or a question mark (?). For example:

     #$keyname            
     #[keyname]            #{keyname}

     #$keyname/elementid   #[keyname]/elementid  #{keyname}/elementid

     #$keyname#mapid       #[keyname]#mapid      #{keyname}#mapid

     The
     concern is that starting the values with a pound-sign (#) or question mark
     (?) will be confusing to users.  There is also a feeling that having
     two pound-signs in a single reference value, while legal, is likely to be
     confusing.  We would like to get confirmation from someone who knows more
     about schemes and URIs then we do, about how this solution will be viewed,
     what is or isn’t legal, and what alternative approaches are possible.
      We are concerned that we have gotten ourselves into a spot were we
     only have mediocre choices.  We’d love to learn that we are mistaken.
     One solution is to omit the leading # or ? character:

     $keyname            
     [keyname]            {keyname}

     $keyname/elementid   [keyname]/elementid  {keyname}/elementid

     $keyname#mapid       [keyname]#mapid      {keyname}#mapid

     If this approach is
     taken, a DITA reference that contains a key reference will no longer be a
     valid URI, although it will resolve to a valid URI.

 
The
     original proposal allowed matching on topic id if there is no match on a
     key defined in a map. Matching on topic ids has been eliminated from the
     current proposal.  Is this OK?

 
Indirect
     references are only supported when the reference attributes have been
     setup to use key references in advance. Do we want to allow information in
     a map to override references that do not contain explicit key references?
     This would make it easier to redirect references without having to go back
     and retrofit key references in existing topics and maps.

 
Do we want to require key names to be valid
     NMTOKENs?

 
Will there ever be a need to defer
     resolution of key references from composition time to viewing time?  If
     so, we may need to rethink the idea of using the current definition of a
     key at the point where the key reference is encountered.

 
Should a new topicref
     specialization, keydef, be defined with a usage attribute value of
     “properties” and the standard usage value for topicref changed to “content
     relationships”?

 
The
     order of priority for combining attributes between key reference and key
     definition elements is patterned after the similar behavior for conrefs
     where the source overrides the target. Or said another way the referencing
     element overrides the referenced element. This means that attributes on
     the key reference element override attributes on the key definition
     element. But for the reference attribute  itself the key definition
     element is overriding the key reference element. Would it be better for all
     of the attributes to follow the same priority order?