Hi Everyone,
As suggesting in yesterday's meeting, I'm sending out
some mails, refereshing everyone's mind on the state of these
discussions.
There was actually some discussion of 119 during the
call on Sept. 2, and maybe it is a good idea to have the discussion
reflected in an email thread, so I will repeat it here.
There question was (from Blaise on behaf of Radu), why
we need to specify this behavior, why can't it be left as a vendor
extension? After all, the resolution to SDO-66 defined strict
compatibility rules, but allowed implementations to weaken them. Can't the
compatibility between keys and entities be allowed under this clause, without
requiring vendors to implement this behaviour?
I would argue the answer is "no", and I'll
give 2 reasons. First of all, the proposal includes more than just the
rule that entities are compatible with their key types, there is a certain
amount of behavior that goes along with it, eg, changes to the values of
keys do not change the value of the key field in the underlying entity, but
change the target of there reference. It's difficult to imagine that this
behaviour would be "compatible" to the behavior of an implementation
that did not support projection between keys and
entities.
The second reason is that this behavior is necessary
for projection to achieve its primary goal, allowing different containment
structures to be imposed on object oriented models. For
instance, consider the m:n relationship between students and courses
in the example. When generating the XML, we want to produce a
document in which Student contains (a reference to) Course. The projection
works exactly because of the special behavior associated with keys. If
Student contained the entity "Course", then we'd get an illegal graph, because
each "Course" would have multiple containers.
Moving data between object-oriented and
document-oriented applications, for instance, between JEE and BPEL,
is perhaps *the* central problem we are looking at SDO 3 to
solve. Achieving this goal requires that the proposal (or one like
it) be accepted. As such, the proposal is too important to be left as a
vendor extension.
Ron
Von: Barack, Ron [mailto:]
Gesendet: Dienstag, 26. August 2008 11:21
An:
Betreff: [sdo] ISSUE 119: Projection and
Keys
Hi
Everyone,
Moving a
discussion from another thread to this thread, for clarity, and so we can
remember the issues:
Blaise
asked:
* What is the impact of relaxing the
SDO name uniqueness requirement on projection?
* What is the impact of SDO-82 on
projection?
Right now, the projection algorithm
matches by name only. It could be that we need to make this a little more
precise when considering that property names may not necessarily be unique, and type names (in the case of XSD anonymous types)
may not even be defined. It's also possible that we can say that
projection, like static SDO generation, has some additional constraints, such as
name uniqueness
These are all valid questions to
consider, but what exactly is the relationship with the proposal being discussed here? Does
having the compatibility rule regarding keys and entities make the above
issues any worse or
better?
* Is SDO-119 useful without an identity
concept? Currently my thinking is that identity management is required here.
There actually is some identity management within the
proposal, the scope in which the identity is meaningful is the graph being
projected. SDO doesn't have any sense of a scope that is smaller than
the HelperContext yet bigger than a DataGraph. Other frameworks, notably
JPA and similar persistence frameworks, do. For example, in JPA, you have
the users persistence context, in which the scope of the mapping between
objects and identity can be managed on a per-thread basis. In SDO,
however, the next larger scope for the identity is the context, and I think we
agree that the context can shared between threads, at the enterprise
application level. It could be that we need to add something like a
persistence context to SDO, but I've been trying to avoid this, and instead to
allow the user to inject his own flavor of entity management by specifying an
EntityResolver... please see my emails to Bryan in this thread, and my earlier
proposals in SDO-125. The current proposal regarding keys and
projection is meant to define a default behavior and a baseline on which the
future EntityResolver functionality can be built. Would you be more
willing to resolve this issue if I added the EntityResolver back
in?
* Alignment with SDO-135. I believe the spirit of SDO-119
& projection in general is to enable alignment with other standards. I
believe these issues should be worked on in parallel, for example the Binder
proposal could be enhanced to unify SDO-135 and projection.
There
were some proposals in this direction, which we moved away from based on your
previous wish that the functionalities be separated. For the earlier
discussion, please see
http://www.oasis-open.org/apps/org/workgroup/sdo/email/archives/200808/msg00001.html
http://www.oasis-open.org/apps/org/workgroup/sdo/email/archives/200807/msg00034.html
The
question where I was hanging is whether or not Binder is allowed to
wrap the POJO or DOM structure, or must copy involve a copy
step. Up to now, we've always though of projection as not requiring a
copy step. If we want to bring the concepts together then we must either
say Binder allows wrapping, or Projection does not allow wrapping. I'd be
willing to further pursue these proposals, but I need some guidence from the TC
as to which direction is appropriate. I actually started work on this
proposal in 119 once I had the impression that we had a better chance of
making progress by separating Binding and Projection. Now the opposite
opinion is being expressed. I usually do not propose something without
first prototyping, so that it is a significant effort to make a
proposal. It is difficult to make progress without stakes in the ground,
such as this one, that provide foundations upon which futher proposals can be
based.
That
said, opinions on wrapping/ copying are more than welcome (but in the
appropriate thead, please)
Ron
Von: Bryan Aupperle [mailto:]
Gesendet: Freitag, 15. August 2008 17:40
An:
Betreff: Re: AW: AW: AW: [sdo] ISSUE 119:
Projection and Keys
I would be happy to address it
separately since I think it opens some interesting questions that have DAS
implications, (Questions like: Does SDO know what DAS is associated any
given type? Does SDO retrieve entities from the DAS directly, or is that
the responsibility of client code? )
Perhaps the best way to handle it would be to make sure we open a
separate issue as we resolve this one.
Bryan Aupperle, Ph.D.
STSM, WebSphere Enterprise Platform Software
Solution Architect
Master Inventor
Research Triangle Park, NC
+1 919-254-7508 (T/L 444-7508)
Internet Address:
"Barack, Ron"
<>
08/15/2008 11:05 AM
To
Bryan Aupperle/Raleigh/IBM@IBMUS,
<>
cc
Subject
AW: AW: AW: [sdo] ISSUE 119:
Projection and Keys
Hi Bryan,
In th current proposal, the
implementation would create one entity for each distinct key value. The
entity would have all default values, except for the key properties, which would
of course match the key.
In earlier versions of the proposal, I defined a
second projection method that took a second argument. I called this
argument an "EntityResolver" and the idea was exactly to allow the client
(probably in this case a DAS) to provide some special logic to manage the
reference objects. In the SAP prototype, for instance, we have an
implementation that basically wraps a JPA EntityManager.
This is certainly
something I want to address before we go final, but I don't think we necessarily
have to define it as part of this issue. It seems to me that the current
proposal is clear enough that it can be accepted as is...we've been unable to
make progress on this proposal for weeks now, and I thought it might go easier
if we added things piece by piece.
One of the things holding me back from
making a proposal with an EntityManager is that even though I think the
functionality is clear and clearly needed, I'm not sure about how the API should
look. Specifically, depending on how ISSUE 134 is resolved, we may get
some kind of "Projector" object, which would be a much better place it
parameterize projection...with an entity manager, or anything else. I'm
not very enthusiastic about having a second "project" method.
It also sort of
disturbed me in our implementation that the application would get back a graph
in which the entities at the edges were attached, but the real SDO content was
still detached. It might be better to allow other hooks into the
projection process, eg, one for normal objects, so that the implementation can
call "merge" or something.
It seems you agree we need an EntityResolver, do
you need to see it as part of the resolution of this issue, or do you think we
can solve it seperately?
Ron
Von: Bryan Aupperle [mailto:]
Gesendet: Freitag, 15. August 2008 16:38
An:
Betreff: Re: AW: AW: [sdo] ISSUE 119:
Projection and Keys
Ron,
I must not have been sufficiently clear about
the scenario I am concerned about.
I understand the case where you start with the
context with entities and project to the context with keys. The mapping
between entities and keys can be created and maintained. I agree there is
no problem projecting back tot he context with entities.
Now consider the opposite case. Start with
an object containing keys and the mapping between entities and keys has not yet
been created (perhaps the object was received via a service call).
Now project to a context with entities. Those entities have to
be retrieved from the data store. Or am I missing something
here?
Bryan Aupperle, Ph.D.
STSM, WebSphere
Enterprise Platform Software Solution Architect
Research Triangle Park,
NC
+1 919-254-7508 (T/L 444-7508)
Internet Address:
"Barack, Ron"
<>
08/15/2008 09:58 AM
To
Bryan
Aupperle/Raleigh/IBM@IBMUS,
<>
cc
Subject
AW: AW: [sdo] ISSUE 119:
Projection and Keys
Hi
Bryan,
good to know that someone is reading the proposals...
I didn't mean
located within the backend store, I meant within the graph reachable from Oe,
called Gk in the text.
Projection maintains identity (one reason why I think it
can be unified with binding). That is, if I have an object O in
HelperContext hc1, and I project it to HelperContext hc2, then
O ==
hc1.project( hc2.project( O ))
What I really what to say in the proposal is
that the same must hold true for keys. That is, if the O has a property p,
that is a DataObject in hc1, and a key in hc2, then
o.get(p) ==
(hc1.project( hc2.project ( O ))) .get(p);
Moreover, whereever I
set the value of a property in the transitive closure reachable from
hc2.project(O) (Gk in the text), to the value of hc2.project(O).get(p), then,
when I project back into hc1, I will always get the same instance, namely
O.get(p).
It's actually easy, it's just hard to say it.
Basically, it means that
when you project from a context with entities to a context with keys, the user
has implicitly defined a map from entities to keys based on the objects in the
projection. This map is not universal... it's based solely on the graph
being transfered.
Did that help or make things worse?
Ron
Von: Bryan Aupperle [mailto:]
Gesendet: Freitag, 15. August 2008 15:25
An:
Betreff: Re: AW: [sdo] ISSUE 119: Projection
and Keys
This
is a tangential question, and perhaps one that I should already know the answer
to, but here goes. In your example with context Ce and Ck, if I have
object Ok and project it into Ce, then for each key in Ok the corresponding
entity has to be located. Since we have agree that entity management is a
DAS responsibility and not an SDO library responsibility, are we laying any new
requirements on DASs, or has the DAS group already agreed that there has to be a
standard API for returning an entity given a key?
Bryan Aupperle, Ph.D.
STSM,
WebSphere Enterprise Platform Software Solution Architect
Research
Triangle Park, NC
+1 919-254-7508 (T/L 444-7508)
Internet Address:
"Barack, Ron"
<>
08/14/2008 07:20 AM
To
<>
cc
Subject
AW: [sdo] ISSUE 119:
Projection and Keys
Hi
Everyone,
I've modified the proposal a bit, based on Frank's suggestion that
projection from keys to entities be made more intelligent. In fact, I
think that the behavior is now intelligent enough so that a second project
method giving a "restoreContext" is no longer required. New text is in
blue.
Comments of course are very welcome.
Ron
Use Cases for Keys and Projection
We begin with
a motivating use case.
Domain models tend to be larger and complex than the views
used by individual clients. Moreover, when going from a large server-side model,
it is necessary to define the scope of the data which should be transmitted to a
remote client. The boundaries of the data which should be transmitted in a
single packet must somehow be defined. On the other hand, the door must be left
open for clients to further explore the model in succeeding calls to the
server.
Imagine a service that returns an Employee object. In
the domain model, Employee has a refence to Department, and Department in turn
has a list of all Employees. If we define the transmission packet to be the
transient closure reachable from Employee, then we can never send a single
employee, we will always send the complete list of employees in the
department.
Our approach is to use keys to represent the boundary
points in a transmission packet. It is assumed that clients that wish to explore
the model beyond these boundaries will be able to use the keys to somehow
perform lookups, but the API through which this is done is out-of-scope for SDO
(being more of a DAS issue).
As an example, we can imagine that the server has a
SDO type system in which Employee has a property with type Department. In the
client's type system, however, the corresponding property has type
{commonj.sdo}string. This value would represent the identity of the Department.
The basic functionality we wish to achive is to allow projection between these
two HelperContexts.
Keys and Projection
A type and its
keyType are compatible with each other. That mean, an object with a
reference to an entity in one context can be projected onto an object with a
reference to the key type in another context.
We illustrate this with an example.
HelperContext #1
Type Employee
*
property - id (Integer) - KEY
* property - name (String)
* property -
direct-report (List of Employee)
is compatible with the metamodel
HelperContext #2
Type Employee
*
property - id (Integer) - KEY
* property - name (String)
* property -
direct-report (List of Integer)
Here we see the type of the direct-report property
has been replaced in the second HelperContext with the corresponding key
type.
The projection from an entity to its key has some
important semantic differences when compared with projection between entities.
Projection between entities creates a new view of the same underlying
data. By contrast, a key does not represent a view, but rather a reference
to an entity in the underlying data model. When working with different
views of the same underlying data, it is natural that changes in one view are
reflected in all other views. By contrast, changing the value of a key
changes the target of the reference rather than the value of the key property in
the referenced data object. When projecting from an entity to a key, a new
instance of the key is always created. By contrast, when we project from
entity to entity, we get the same number of objects in the target HelperContext
that we had in the source HelperContext.
Conversely, when projecting from a key to
an entity then for each distinct key value within the graph being projected, all
references to that key must resolve to the same entity.
In many scenarios data will round-trip
between contexts, including between contexts in which entity map to keys.
Let us consider two context Ce and Ck, representing the entity and the key
context, respectively, and a DataObject Oe, in context Ce. Projecting Oe
into Ck returns a DataObject, Ok. The transitive closure reachable from Ok
is Gk. Every key value in Gk maps to an single entity in Ce, and it is
this entity is that is found when Ok (and effective, all of Gk) is projected
back into Ce. In cases where the user has set a key property to a value
that is not found in Gk, then, as a result of projecting Ok into Ce, a new
entity will be created. The created entity has default values for all
properties other than the key fields.
We illustrate with an example. Note that the
example uses containment relationships in both contexts. This is done for
clarity, since it allows an XML representation of the data. The use-case,
however, is stronger when the contexts have different (or perhaps no)
containment structures. Imagine the following data in HelperContext
#1
<employee id="11">
<name>Foo Bar</name>
<direct-report
id="21">
<name>Jane
Doe</name>
</direct-report>
<direct-report id="31">
<name>Jim Jones</name>
</direct-report>
<direct-report
id="22">
<name>John
Smith</name>
</direct-report>
</employee>
After projecting to HelperContext #2, we have
the following data
<employee id="11">
<name>Foo
Bar</name
<direct-report>21</direct-report>
<direct-report>31</direct-report>
<direct-report>22</direct-report>
</employee>
If we imagine the client changes the
list of direct reports, so that the second item in the list has value “41”
instead of “31”, then the meaning of the change is not that the employee with
name “Jim Jones” now has a new ID, but that “Jim Jones” has been replaced by
another employee.
On projecting from context 2 back into context 1, the
instance of Employee with id="22" would be detached from the Employee with
id="11". Also a new Employee with id="41" would be created and added as a direct
report to the Employee with id="11". If the graph in context 1
were in scope of a ChangeSummary, then because the "direct-report" property in
context1 is a containment property, the change would be tracked as a delete or a
create. If the relationship were non-containment, the change would be
tracked as a modification to the employee with id=11, not as a delete to the
employee with id=22 or as the creation of a new employee with id=41;
semantically, it is only the reference to entity 22 that has changed, the entity
itself still exists, and is unaltered.
Projecting this data from context 2:
<employee id="11">
<name>Foo Bar</name
<direct-report>21</direct-report>
<direct-report>41</direct-report>
<direct-report>22</direct-report>
</employee>
Into context 1 yields:
<employee
id="11">
<name>Foo Bar</name>
<direct-report id="21">
<name>Jane Doe</name>
</direct-report>
<direct-report
id="41"/>
<direct-report id="22">
<name>John
Smith</name>
</direct-report>
</employee>
Projection between entities and keys
becomes more powerful when the relationship through which the entities tie into
the graph are non-containment. The following example shows how a complex
model that lacks containment relationships can be projected onto a context that
requires a specific XML serialization, which implicitly prunes the orignal
domain to the requirements of a specific client.
HelperContext #1
Type School
*
property - name (String) - KEY
* property - students (Student) -
many=true
* property - courses (Course) - many=true
Type Student
* property - name (String) - KEY
* property - courses
(Course) - many=true, containment=false, opposite=students
* property -
school (School) - containment=false, opposite=students
Type Course
* property - name (String) -
KEY
* property - students (Students) - many=true, containment=false,
opposite=courses
* property - school (School) - containment=false,
opposite=courses
Notice the m:n relationship between
Student and Course. If we imagine a service that should expose this domain
model to clients, it is possible that some clients will wish to obtain the list
of students participating in a particular course, while other clients may wish
to obtain the list of courses in which a particular student is enrolled.
The application cannot determine based on the structure of the data which
of the two possible containment structures is "correct". Notice also that
returning the transitive closure would return all the data associated with the
entire school.
In this example the client wants the data
structured according to the following XSD
<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://www.w3.org/2001/XMLSchema" targetNamespace="http://projection"
xmlns:tns="http://projection"
elementFormDefault="qualified">
<complexType name="School">
<sequence>
<element name="students"
type="tns:Student" maxOccurs="unbounded"/>
</sequence>
<attribute name="name" type="string"/>
</complexType>
<complexType
name="Student">
<sequence>
<element name="courses" type="string" maxOccurs="unbounded"/>
</sequence>
<attribute name="name" type="string"/>
</complexType>
<element name="school"
type="tns:School"/>
</schema>
Notice that we are imposing a containment
structure on the original context, as well as pruning it by replacing the course
entity by the corresponding key. The following code illustrates the
behavior of the project method.
DataObject cal =
_helperContext.getDataFactory().create(School.class);
// Create the SDO graph
cal.set("name","Berkeley");
DataObject billy =
cal.createDataObject("students");
billy.set("name", "Billy Bear");
DataObject bob =
cal.createDataObject("students");
bob.set("name", "Bobbie Bear");
DataObject
basketWeaving = cal.createDataObject("courses");
basketWeaving.set("name",
"Basket Weaving");
DataObject algol = cal.createDataObject("courses");
algol.set("name",
"Algol");
DataObject revolution =
cal.createDataObject("courses");
revolution.set("name",
"Revolution");
// hook things up
billy.getList("courses").add(basketWeaving);
billy.getList("courses").add(algol);
bob.getList("courses").add(basketWeaving);
bob.getList("courses").add(revolution);
// Create a second context
defined by an XSD
HelperContext hc2 = HelperProvider.getNewContext();
hc2.getXSDHelper().define(getClass().getClassLoader().getResourceAsStream("com/sap/sdo/testcase/internal/pojo/ex/projection.xsd"),
null);
// Project from the java context to the XSD context
DataObject projection =
hc2.getDataFactory().project(cal);
// Produce XML based on the
XSD
String
xml = hc2.getXMLHelper().save(projection, "http://projection",
"school");
// I'm imagining here that sending the XML out over the wire
(eg, using it as a response to a
// WebService request. On the client side,
we go from the XML back to SDO. We use the context
// based on the
XSD.
DataObject projection2 =
hc2.getXMLHelper().load(xml).getRootObject();
// We can make some changes. We can add a
new course...
projection2.getList("students.0/courses").add("Fortran and
You");
//
So, now the trip back to the server…
//I'm skipping the XML step, and simply
projecting the modified (XML oriented) data back into
// my java context
DataObject cal2 =
_helperContext.getDataFactory().project(projection2);
// Test that there is one entity per key
value
DataObject billy2 = (DataObject)cal2.getList("students").get(0);
DataObject basketWeavingBill =
(DataObject)billy2.getList("courses")
.get(0);
DataObject bob2 =
(DataObject)cal2.getList("students").get(1);
DataObject
basketWeavingBob = (DataObject)bob2.getList("courses").get(0);
assertSame(basketWeaving2,
basketWeavingBob);
