> I don't care about !!, it is just a starting point :)
I figured as much, just trying to avoid the special cases with two symbols.
> You can't specify what order the client sends the XDI statements, so you can't "specify the metadata always be loaded first"
Good point. That limits us then to either data: URI, or multipart XDI messages..which might work but is added order of magnitude in complexity for XDI message in my mind.
>If we specify that binaries are encoded in a certain way using the data URI format, that might be a solution because in one XDI statement, we can determine that we are dealing with a binary.
Agreed. Ease of use at tradeoff of encoding overhead. I recommend we support that as part of core spec, then have an optional profile (after we get spec out) to support multipart XDI messages.
On Fri, 2 Mar 2012, Barnhill, William [USA] wrote:
> To make this work I think we need to first specify how a binary would
> be serialized in the XDI serialization protocol. Since that is based
> on JSON you either must use an encoding (i.e. data URL, or MIME), or
> have some part of multi-part message with the binary as a different
> MIME type. I think we need to specify how to handle binaries, but I
> don't think !! is the answer. I think specifying that the metadata
> always be loaded before the value or a literal is processed, may be
> the answer, which would allow our existing mechanisms for specifying
> the binary MIME type (we still don't have serialization finalized,
> currently encoding is used I believe by several of us).
>
>
> Kind regards,
>
> Bill Barnhill
> Booz Allen Hamilton - Belcamp,MD
> 1-443-924-0824|
>
>
>
> -----Original Message-----
> From: [mailto:] On
> Behalf Of Michael Schwartz
> Sent: Friday, March 02, 2012 10:24 AM
> To: OASIS - XDI TC
> Subject: Re: [xdi] proposal for binary data literal node
>
>
> I want to stress that storage of binaries is not a niche use case. It doesn't matter what kind of implementation you are using--rdbms, ldap, or file system--you will want to treat binaries differently. An implementation detail is what do you do with the binaries, not whether you know if you have a binary.
>
> If we don't solve this, and we're stuck with some lame solutions like encoding binaries (i.e. like in SMTP), it would be a missed opportunity for the TC.
>
> How about a new type of arc: the "binary arc" ?
>
> See inline comments below.
>
>
>>
>> If for optimization purposes a server needs to know the data type of
>> a literal (binary, number, boolean, etc.), then it could either 1.
>> figure out the data type based on the dictionary, or
>
> I don't understand your suggestion here. How would a dictionary be helpful? How could you guarantee that the dictionary is known when you are parsing the XDI statements?
>
>
>> 2. require (as a policy) that clients add the literal together with
>> the metadata that specifies the data type
>
> no, per my previous comment, there is no guarantee the metadata would be loaded first.
>
>>
>> Markus
>>
>> On Fri, Mar 2, 2012 at 3:05 PM, Michael Schwartz <> wrote:
>>
>>>
>>> I think we need a way to differentiate binary literals if we want to
>>> support storage of binaries in the graph. And simply storing
>>> metadata is not sufficient because there is no guarantee that the
>>> metadata is created before the literal needs to be stored. Is there a solution?
>>>
>>> thx,
>>>
>>> Mike
>>>
>>>
>>>
>>>
>>> ------------------------------**---------------------
>>>
>>> Michael Schwartz
>>> Gluu
>>> Founder / CEO
>>>
>>> +1 646-810-8761
>>>
>>>
>>> On Thu, 1 Mar 2012, Drummond Reed wrote:
>>>
>>> Mike, the question you raised actually highlights a number of the
>>> key
>>>> reasons for the literal context proposal. Let me explain.
>>>>
>>>> First, per Markus' last point in his message, one primary purpose
>>>> of literal contexts is so an XDI processor always knows the type of
>>>> the literal node it is dealing with by the type of the literal context.
>>>> To use Mike's example, if "*logo" is defined as a literal context
>>>> containing a binary value (which has several semantic issues -- see
>>>> below), then @!1111*logo always identifies a single binary value.
>>>>
>>>> Secondly, it's a misunderstanding of literal contexts that there
>>>> could be two literal arcs. The whole idea of a literal context is
>>>> that it has a single literal arc. If the literal context is defined
>>>> as containing binary data, then that literal arc points to binary
>>>> data. If the literal context is defined as containing a number,
>>>> then the literal arc points to a number.
>>>> And so on.
>>>>
>>>> These are all controlled by XDI dictionary definitions, which is
>>>> why I think the XDI Dictionary spec is moving up in priority.
>>>>
>>>> Now, the semantic issues. First, @!1111*logo is semantically wrong
>>>> because uses a *name to identify a literal context defined in a
>>>> dictionary, and * is not the dictionary namespace. The dictionary
>>>> namespace is + (with the exceptions of a handful of special $words defined by the XDI TC).
>>>>
>>>> So it should be @!1111+logo. The second semantic issue is that
>>>> +logo should not be a literal context because it is not a single
>>>> defined datatype.
>>>> +logo
>>>> should be a supertype that has defined subtypes. In other words,
>>>> the dictionary defintion should say +logo/$is$a/+image, and (among
>>>> other
>>>> options) +$image/$a/$a$xsd$jpeg. So if the binary you wanted was a
>>>> JPEG, then the address would be:
>>>>
>>>> 1. @!1111+logo$a$xsd$jpeg <== canonical JPEG version of the logo
>>>> (zero-or-one)
>>>> 2. @!1111+logo/$a$xsd$jpeg <== (note the slash) relational arc
>>>> pointing
>>>>
>>>> to all the JPEF versions of the logo (zero-or-more)
>>>>
>>>> So it is the dictionary-defined $jpeg, and not +logo (and
>>>> definitely not
>>>> *logo) that is the XRI literal context storing the binary data. And
>>>> because the $dictionary will define $jpeg as a JPEG binary, any
>>>> time an XDI processor sees $jpeg as the literal context, it knows
>>>> the value is a JPEG.
>>>>
>>>> I'll put this on the agenda for tomorrow's call (even though Mike
>>>> won't be there).
>>>> =Drummond
>>>>
>>>> On Thu, Mar 1, 2012 at 4:35 PM, Michael Schwartz <> wrote:
>>>>
>>>>
>>>>> Markus,
>>>>>
>>>>> Thanks for the quick response!
>>>>>
>>>>> 1. Yes, you could have both a ! and !! arc, why not? It doesn't
>>>>> seem inherently problematic to me.
>>>>>
>>>>> 2. yes, you're right I forgot the / for the literal arc :)
>>>>>
>>>>> 3. !! was just an idea... it could be something else.
>>>>>
>>>>> 4. Its useful to be able to differentiate binary data from other
>>>>> types of data because there persistence strategy varies greatly.
>>>>>
>>>>>
>>>>> - Mike
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 1. Could you have BOTH a binary and a non-binary literal in a context?
>>>>>
>>>>>> I.e.
>>>>>> have a ! arc and a !! arc.
>>>>>> If not, will the server throw an error if you already have one
>>>>>> literal and try to add the other one?
>>>>>>
>>>>>> 2. In your example, wouldn't the XRI actually be @!1111*logo/!!
>>>>>> instead of @!1111*logo!! ?
>>>>>>
>>>>>> 3. Something is bothering me about the fact that !! is two subsegments.
>>>>>> I
>>>>>> am worried it will result in ambiguity in one way or the other.
>>>>>>
>>>>>> 4. I don't understand why you can't load the metadata together
>>>>>> with the literal, to figure out whether it is binary or not.
>>>>>> Wasn't this the whole point of what Drummond called "literal contexts"?
>>>>>>
>>>>>> Markus
>>>>>>
>>>>>> On Thu, Mar 1, 2012 at 9:49 PM, Michael Schwartz <> wrote:
>>>>>>
>>>>>>
>>>>>> XDI TC:
>>>>>>>
>>>>>>> I'm attaching a proposal to represent binary literal values as "!!"
>>>>>>>
>>>>>>> So, for example, the XRI for a my company's logo might be :
>>>>>>> @!1111*logo!!
>>>>>>>
>>>>>>> The fact that the image is a binary can't be represented in the
>>>>>>> metadata, because it is not known which will be loaded first,
>>>>>>> the literal node, or the metadata about the literal node.
>>>>>>>
>>>>>>> This is a critical issue to optimize persistence of binary
>>>>>>> objects, which need to be indexed differently.
>>>>>>>
>>>>>>> I'd like to fast track this because we need to store binaries as
>>>>>>> part of the oxPlus demo :)
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>> thx,
>>>>>>>
>>>>>>> - Mike
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------****----------------------------**
>>>>>>> -
>>>>>>> -**
>>>>>>> ---------
>>>>>>> To unsubscribe, e-mail:
>>>>>>> -****open.org<http://open.org>
>>>>>>> <xdi-unsubscribe@**lists.oasis-open.org<
>>>>>>> s
>>>>>>> is-open.org>
>>>>>>>>
>>>>>>>
>>>>>>> For additional commands, e-mail:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> ------------------------------****----------------------------**
>>>>> --**---------
>>>>> To unsubscribe, e-mail:
>>>>> -****open.org<http://open.org>
>>>>> <xdi-unsubscribe@**lists.oasis-open.org<
>>>>> s
>>>>> -open.org>
>>>>>>
>>>>>
>>>>> For additional commands, e-mail:
>>>>>
>>>>>
>>>>>
>>>>
>>> ------------------------------**------------------------------**----
>>> -
>>> ---- To unsubscribe, e-mail:
>>> -**open.org<-o
>>> p en.org> For additional commands, e-mail:
>>>
>>>
>>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:
>
>
