Hi Mike,
On 11/9/10 1:23 AM, Mike Edwards wrote:
Eric,
Comment #1:
You did not address any aspect of
the
"relatesTo" concept that I discussed in my note.
My apologies. I dealt with it obliquely (or at least I tried to),
but not head on. So let me try to address it head on.
As I see it, if there's some notion of "send further related
messages to destination foo" capability (a.k.a. JMSReplyTo with
JMS), then the "relatesTo" concept (a.k.a. JMSCorrelationID for JMS)
may or may not be needed. To set this up, assume three components
A, B, C, where A & B are sending events, and C is doing some
processing then sending something "related to" the event it
received.
Possibility 1:
A & B are looking for follow-on events from C that relate to
*specific* outgoing messages that A & B sent. Since this is
specific to a message, some unique message identifier, "relates to"
information if you will, has to be carried somewhere - either as the
"metadata" for the event (JMSMessageId/JMSCorrelationId), or as part
of the business data of the event.
In this possibility, the "relates to" information would be key to
further processing by A & B. But I must confess, this isn't the
point of the issue I raised, and I've not thought deeply either
about setting this information, or about accessing it.
Possibility 2:
A & B are looking for follow-on events from C that relate to a
general subject about which A & B send events. In that case,
the business data of the event already contains sufficient
information for A & B to process the results.
In this second possibility, you might think that just because A
& B can process the results based on the content of the message,
that JMSReplyTo plays no role. To clarify, I have a specific
scenario to consider. Please note, I manufactured the following
scenario; any relation to any real use is entirely incidental!
Suppose A & B are something like "stock tickers" (+0.02 XYZ,
-0.03 ABC ...), but for different stock exchanges. Now, component C
centralizes all the buy/sell logic of an organization, but to
simplify implementation, the organization wants to send back a
buy/sell/hold decision to a stock-exchange-specific-component to
act.
If A & B both listen on the same "buy/sell/hold"
channel/destination that C publishes to, then each receives twice as
many messages as it needs to, and drops half of them on the floor
(well, that assumes equivalent volatility and distribution of stocks
being traded, but you see my point). Now suppose that there are
five or ten components like A & B. The system might be
enormously inefficient if each component received 5-10X the number
of messages it actually needs to process. So you won't do that.
You want messages from C about buy/sell/hold to go back to the
stock-exchange specific components that need to act.
Two paths to implementation (that I can think of)
either you bake logic into the "C" component, and that
component decides where to route messages based on the
information in the messages it receives
or you let components A, B, ... specific a "reply to"
destination where C sends whatever information it chooses to
send. No extra logic in component C, and the assembler for the
composite controls the number and character of the
destinations/channels instead of the implementer of component C.
I prefer the second path.
-Eric.
Does this mean that is it not
interesting
/ not required from your viewpoint?
Yours, Mike.
Strategist - Emerging Technologies, SCA & SDO.
Co Chair OASIS SCA Assembly TC.
IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great
Britain.
Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431
Email:
From:
Eric Johnson
<>
To:
Mike
Edwards/UK/IBM@IBMGB
Cc:
OASIS Assembly
<>
Date:
09/11/2010 02:02
Subject:
Re: [sca-assembly] Re:
[ASSEMBLY-249]:
Need some notion of "callback" address in conjunction
with eventing
Hi Mike,
I agree, an inline response might further complicate matters, so
let me
see if I can take your points, restate them, and see what we can
discuss,
and see what I can clarify.
MJE: The one important point with references is that the client
KNOWS the
number of responders
EEJ: Agreed. The scenarios I'm trying to address do not assume
that
knowledge.
MJE: To me, the EXPECTATION of a particular response implies
service-like
behavior.
EEJ: Agreed. What I'm getting at is the expectation of the
possibility
of some kind of "response", without knowing where the
"response",
if any, is coming from, and what form that "response" will take,
or even whether, strictly speaking, you'd call it a "response."
MJE: The "ReplyTo" notion indicates the expectation of a
particular
pattern of messages, which I think is actually modelled in SCA
using services/references
with one-way messages and callbacks.
EEJ: The word Danny used was "deconstructionist." This
is but one possible way to use "ReplyTo". As I mentioned
in my email, the mistake JMS makes, which WS-Addressing doesn't
make, is
to assume one particular metadata label for this construct. In
90%
of the use-cases, that single piece of additional metadata,
however employed,
can serve a valuable role, even if, strictly speaking, it isn't
a "reply"
channel.
MJE: Well, that is an important point - it may well be the case
that the
message business payload carries some identification that can
later be
used for "relatesTo" information in subsequent events that are
generated.
EEJ: The trouble is, if you always force that back upon the
application/business
logic, then the business logic may need to make decisions about
where to
send follow-on messages. However, if you let the incoming
message
dictate transport level details about where to send follow-on
events, then
the business logic doesn't need to care.
MJE: For this feature to be of any real use, then it is an
application-to-application
communication function. In other words, when a component
produces
an event and sends it out into the world, the component itself
MUST have
the unique ID for that event (either generated by the component
itself
OR by some API involved in the produce process). Equally, when
an
event is consumed by some other component, that component must
be able
to a) know that there is a unique ID for the event and b)
obtain
that ID
EEJ: I don't agree that this is necessarily an application to
application
communication function, at least not in the way that you
describe. Component
A sends an event via some channel X, which happened to arrive at
Component
B, with the request that subsequent "related" messages get sent
to channel Y, which might end up at Component C. Likewise
Component
D sends a message to channel X, and it again happens to land at
component
B. However D requested that "related" messages get sent
to channel Z, and that happens to land at Component E.
Presumably
both Component C & E know the subject of the events that
they receive
- they do not need to know the transport details that led them
to receive
those messages. That would be extra coupling!
MJE: One thing to note about this is that it makes *NO*
assumptions about
who receives any events with "relatesTo" metadata. This
may mean that "relatesTo" may be meaningless for some consumers
of an event, but I see no harm in that.
EEJ: Agreed!
MJE: The bit I don't buy is the notion that "replyTo" seems to
indicate a direct connection of some kind back to a single
component only
(A) for the events produced by B, C, D in response to an initial
event
from A. Why can't assemblers choose to direct those events
wherever
they please?
EEJ: I think they should be able to as well. Both you and
Peter,
if I remember correctly, seem to have been confused by my random
mutterings,
thinking I want "replyTo" to come back to the particular
component,
under the control of the component. In both cases, I'd say no.
I
fully expect that the assembler decides the response
destination, and that
may be whatever the assembler chooses, and it might be the
original component,
or it might not. Notably, it might be to another channel.
MJE: Ahem, I beg to differ on the interpretation here - far from
"loose
coupling", this "replyTo" notion is an introduction of much
tighter coupling between 2 components.
EEJ: I don't see a better, easy way to address the scenario I
put above,
where component B sends follow-up events based on decisions by
the assembler,
because of where events to component B originated. I don't see
how
you can do this with current services and references. I don't
see
how you can do it with current eventing.
You might be able to get something to work by injecting
properties
into a components A & D so that it can augment the business
data to
indicate where B is supposed to send follow-on messages, but
that further
requires that B knows that it has to pay attention to additional
business
data to route messages. So at best, the assembler can suggest
an
intention only indirectly, and it relies on a proper
implementation of
B to manage message routing. All of which is unnecessary, since
all
of the possible bindings could support it.
So yes, you are introducing an unnecessary tighter coupling if
you don't
support *something* additional here.
MJE: You will need to convince me on point #4 - I don't see
"tight
coupling elsewhere", neither do I see inefficiencies.
EEJ: To be fair, one further way to solve the scenario I
outlined above
would be for components C & E to listen to a known channel.
However,
I think this now implies, (a) decisions at a business logic
level, (b)
additional messages processed by both C & E. It is simply
less
flexible.
MJE: SCA can use the wire protocol features when they are
useful, but only
as a reflection of the composition goals that are already shown
in the
SCA assemblies.
EEJ: I agree. I don't want to consider this as specific to the
JMS
notion of JMSReplyTo. I want to solve something like the
scenarios
I've outlined above. What little research I did into this
suggests
the possibility of any possible "eventing" capable transport
of carrying a notion of "subsequent destinations". Call
those whatever you wish. JMS happens to (narrowly) call this a
"reply
to". But as near as I can tell, that is the 90-99% use case.
So I think we should be able to leverage this transport
capability
in SCA, but that means providing some means to model, in the
assembly,
some way to wire something that maps to this transport level
property.
Not supporting this notion feels like composing an eventing
system with
one hand tied behind your back.
Hopefully that brings some clarity to the points.
-Eric.
On 11/8/10 2:37 AM, Mike Edwards wrote:
Eric,
I'll try one further inline response, but I'm concerned this
will get very
messy.
Yours, Mike.
Strategist - Emerging Technologies, SCA & SDO.
Co Chair OASIS SCA Assembly TC.
IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great
Britain.
Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431
Email:
Eric Johnson <>
wrote on 04/11/2010 23:01:27:
> [image removed]
>
> Re: [sca-assembly] Re: [ASSEMBLY-249]: Need some notion
of
> "callback" address in conjunction with eventing
>
> Eric Johnson
>
> to:
>
> Mike Edwards
>
> 04/11/2010 23:06
>
> Cc:
>
> OASIS Assembly
>
> Hi Mike,
>
> It is possible we're vehemently agreeing on many points.
Which
> should make this an interesting discussion....
>
> On 11/4/10 9:11 AM, Mike Edwards wrote:
>
> Folks,
>
> This message from Eric starts to get into one of the
things that
> Danny raised on he call earlier this week, namely the
> inability of SCA to model the various messaging patterns
describable
in WSDL.
>
> I DISAGREE that this is the case, and I will describe how
SCA can
> deal with these cases.
>
> > * message out, maybe message back
> > * message out, N messages back
> > * message out *, message back
>
> Currently, SCA can do ALL of these using references &
services,
> using Callbacks.
>
> I agree, to the extent that you're talking about a single
consumer
> at the destination. Concretely mapping this to JMS (or
email),
all
> I know is that I send messages to a destination, and I
don't know
if
> I get a "message" back at all, whether I get five
"messages"
back,
> or just one, and I further don't know if those "messages"
will come
> from the same source.
>
> Where a service has both an interface AND a callback
interface, then
> it is saying:
>
> - for any forward invocation of the service
> -- there may be 0, 1 or many invocations of any one of
the
> operations in the callback
> interface (which can also be read as "response
messages")
> - the invocation of an operation on the callback
interface can occur
> in response to one forward invocation
> -- or to some sequence of multiple forward invocations
>
> Yes, all true. The way I read it, you've assumed a
single
> "responder", whereas I have not.
SCA references provide for 0..n and 1..n multiplicity which
gives as many
responders
as you wish. The one important point with references is
that the
client KNOWS the
number of responders.
>
> I perceive an important semantic distinction between
"here's
a
> message", and "invoke particular operation", even if
they're called
> with the same data. Eventing systems move data, whereas
"services"
> systems perform actions based on parameters. I think
they align
> about as much as REST does with SOAP. You can accomplish
the
same
> ends with both, but with REST you think primarily about
resources,
> representations, and state, whereas with SOAP you think
about verbs
> and parameters. You can end up in radically different
places,
with
> radically different architectural implications. In my
head at
> least, that analogous distinction extends to
messaging/eventing vs.
> web-services/SOAP.
Well, yes and no.
To me, the EXPECTATION of a particular response implies
service-like behaviour.
Event style interactions can make no assumption at all about
there being
any
kind of response to a given emitted event, even where
multiple consumers
receive the event.
The "ReplyTo" notion indicates the expectation of a
particular
pattern of
messages, which I think is actually modelled in SCA using
services/references
with one-way messages and callbacks.
I harp on about this because some while ago we discussed the
whole question
of
why we could not model all of event processing using this
technique of
one-way
messages with callbacks... and we decided that event
processing was different
in a number of significant ways.
>
>
> > * message out, message back to different component
>
> - this is supported through an override of the callback
binding
> HOWEVER this is then rightly an aspect
> of the configuration of the ASSEMBLY and not the result
of some
> individual implementation artifact
> deciding on the wiring - ie there is no way that the
"client"
> component should itself be dictating
> the target for the response - that MUST be part of the
composition/assembly.
>
> I agree completely.
>
>
> Now, I think the real debate here is whether "replyTo"
has
**ANY**
> place in the style of Event Processing
> that the SCA spec tries to describe.
>
> Looking forward in your email, you through out a notion
of
> "relatesTo". That *exactly* what I'm trying to get
at with this issue.
>
> In certain circumstances, the "relatesTo" meta-data
exists
as part
> of the application-defined payload of a message, and
that's probably
> unavoidable. I don't want to aim to boil the ocean, as
it were.
Well, that is an important point - it may well be the case
that the message
business payload carries some identification that can later
be used for
"relatesTo" information in subsequent events that are
generated.
>
> I'd like however, if the binding can handle the
"relatesTo"
notion
> whenever it makes sense. That raises exactly the
question I'm
> trying to get at - how does the binding even *know* that
it is
> supposed to manufacture a unique identifier so that a
subsequently
> delivered message can be correlated via some notion of
"relatesTo"?
> When that subsequent message does arrive, how does the
binding layer
> even know to associate it with whatever state it might
have stored
> earlier? How did the first, second, or third consumer
know where
to
> send the "relatesTo" information?
Well, this is *not* a bindings question, first &
foremost. For
this feature
to be of any real use, then it is an
application-to-application communication
function. In other words, when a component produces an
event and
sends it
out into the world, the component itself MUST have the
unique ID for that
event (either generated by the component itself OR by some
API involved
in the produce process). Equally, when an event is consumed
by some
other
component, that component must be able to a) know that there
is a unique
ID
for the event and b) obtain that ID
There would then need to be a further step for the receiving
component
so that
when it produces some other event(s), that it can "label"
one
or more of those
events with a piece of "relatesTo" metadata, which can be
retrieved
by whichever
components receive that event.
One thing to note about this is that it makes *NO*
assumptions about who
receives any events with "relatesTo" metadata. This may
mean that "relatesTo"
may be meaningless for some consumers of an event, but I see
no harm in
that.
>
> How can the consuming binding of a message receive the
necessary
> information from the incoming message and stash it, such
that when
> the consumer then produces a message that relates to the
originally
> received message, it will work? The code calling the
producer
must
> either get at the "relatesTo" information, or tell the
producer
it's
> invoking to look up that previously saved state....
Now, I've used the dreaded word - "METADATA" - and I've used
it deliberately.
These pieces of information - eventID and relatesTo - I
regard as metadata.
We've made provision for there to be metadata related to
events, but so
far
we have not taken the step to standardize any metadata.
Do the functions outlined here represent metadata that we
should seek to
standardize? (And for the client APIs in language X, some
functions to
manipulate said metadata???)
>
> The real question that Eric & Danny need to address
is how an
SCA
> event producer can make **any** assumption
> that the receiving component actually produces any kind
of response
> message. A subsidiary question is whether
> the event producer can also make some assumption that the
receiving
> component will **only** send some message
> back to the event producer and nowhere else.
> I don't think I've made either assumption. The only
assumption
I
> want to make is that an assembler can *choose* to do so
with event
> producers and consumers that declare that they have an
> implementation that leverages the notion of "relatesTo"
or "replyTo".
>
> I think that our current model of event processing is
simply more
> loosely coupled than this. If an event
> producer anticipates that some new events may be
generated as a
> result of some events that it produces, then
> I would say that the component with the producer should
also have
a
> consumer for that type (or types) of event.
> I can think of use-cases where that doesn't work. For
example,
> assume I have a component A that produces messages on
some global
> domain channel //foo. At some point, some set of
components
show up
> in the system, and start receiving messages on //foo, and
perhaps
> some of those components (call them B, C, D) start
sending
> "relatesTo" information about messages originally sent to
//foo.
>
> How can component A catch these "relatesTo" messages?
Option
(1) -
> listen to a predetermined channel //bar (or //foo?) where
those
> messages are expected to be sent. Fails if the assembler
doesn't
> wire producers on B, C, and D properly, and
unfortunately, this key
> bit of information *isn't* in the model. Option (2) - A
consumes
on
> the default domain channel "//". Problematic in that
hosting
> environment of A, if not A itself now sees far more
messages than
it
> needs to, in order to find the ones it does want. Option
(3)
- as
> part of the originally produced message, send an
indication of where
> the "relatesTo" messages ought to be sent. This last
approach works
> even if A, B, C, and D are deployed in different
composites and at
> different times.
Well, I'll start by saying that components don't turn up
randomly in the
system and if the assembler isn't in control then all kinds
of crazy things
may occur. Option (1) seems a reasonable approach. I'd
certainly
expect
the assembler a) to know that B, C, D actually produce these
types of event
and b) to know that those events need to be sent somewhere.
The fact that
A
consumes events of that type is a bit of a hint...
The bit I don't buy is the notion that "replyTo" seems to
indicate
a direct
connection of some kind back to a single component only (A)
for the events
produced by B, C, D in response to an initial event from A.
Why can't
assemblers choose to direct those events wherever they
please?
Oh, and by the way, are you saying that "direct connections"
from producers
to consumers (without involving some intermediate channel)
are a good thing
and should be (re)introduced into the model??
>
> It is then up to the assembler to decide to connect both
the
> producer and the consumer separately to appropriate
> channels - similarly for any "receiving" component's
consumer
& producer.
>
> I absolutely agree that the actual choice of how to
handle
> "relatesTo" messages should be addressed by the
assembler.
I'm just
> noting that by not supporting the notion of "replyTo",
we've
taken away
> one option for loosely coupling.
Ahem, I beg to differ on the interpretation here - far from
"loose
coupling",
this "replyTo" notion is an introduction of much tighter
coupling
between 2
components.
>
> Any "correlation" needed between an initial event and
some
future
> "response" event should be managed through
> some aspect of the response event (eg a "relatesTo"
field).
>
> Indeed yes, if we choose not to model these details, then
SCA users
> will need to force such correlation information into the
> application/"business" layer, rather than letting it be
handled at
> the binding layer. Or they'll choose a vendor
implementation
that
> extends SCA to enable some amount of correlation to be
done at the
> binding layer.
>
>
> My other reaction to this material is to say "if you want
a
> response, use services/references, not event processing"
>
> I understand that reaction. I can see how what I'm
saying feels
> like it falls into the category of "if all you have is a
hammer...".
> But I think what I've offered above makes what I'm
arguing for
> different in several ways:
>
> 1) Eventing is about moving data, not invoking
operations, which I
> see as a key semantic difference
> 2) I'm not assuming a single "responder", nor a single
message
back,
> nor a response that comes back even to where it started
from. We're
> talking about choreographed message exchanges, rather
than
> individual message exchanges. Trying to model this with
WSDL,
or
> even with two WSDLs, is an effort in futility. Even
worse, trying
> to pretend that WSDL captures these choreographies
obscures the fact
> that the actual message flows aren't doing the standard
request/
> reply exchanges that WSDL typically models.
> 3) By not supporting some model notion of
"relatesTo"/"replyTo",
> end-user applications will be forced to move such data
into event
> messages, mixing transport and business data
unnecessarily, rather
> than having the binding take care of it.
> 4) By not offering some way of indicating a "reply"
destination,
> that moves a tight coupling elsewhere in the system - in
the form
of
> assumed knowledge about where those events will be sent,
or assumed
> knowledge about the types of events will be sent, or
both. Or
you
> build in unnecessary inefficiencies.
You will need to convince me on point #4 - I don't see
"tight coupling
elsewhere", neither do I see inefficiencies.
>
>
> As to the question of why the various protocols listed
below have
> "replyTo" - I say it is because those protocols have
> to be able to deal with request/response message patterns
of the
> type listed above - the current SCA event processing
> model AIN'T REQUEST/RESPONSE. It is *much* more loose
coupled
than that...
>
> As I point out in #4 above, I think the association of
request/reply
> with tight coupling is a mental trap of sorts. Just
because
I've
> indicated a reply to address, doesn't mean that I've
tightly coupled
> anything. Rather, I've provided a means for late
binding. The
> world of HTML over HTTP is anything but tightly coupled,
and it is
> built entirely on a request/response paradigm. Some uses
of
HTML
> over HTTP are tightly coupled, but that just demonstrates
that you
> can abuse any system. I think I can assemble some fairly
> interesting message choreographies that will leverage a
"replyTo"
> notion, but that in no way implies that the components
involved are
> tightly coupled.
A "relatesTo" notion, I can buy (see previous comments), but
"replyTo",
if it is taken to mean "any events generated as a 'response'
to this
event MUST be sent to this place" I can interpret in no
other way
than a
tight coupling between the original producer and the
receiving consumer.
"replyTo" seems less than useful in the event processing
model
we've built
for SCA. I don't see the need for it.
>
> I, for one, question why JMS singled out "JMSReplyTo" as
the only
> way to send a Destination to a recipient. It ought to
have been
> more open ended than it is. WS-Addressing doesn't repeat
this
> mistake. But generically, the ability of the event
producer
to
> indicate one or more interesting "Destinations" for
sending
follow-
> on messages seems like a way to further decouple systems,
as it
> saves the recipient of those messages from having to
pre-emptively
> "wire" to those destinations.
The problem with these low-level wire-protocol features is
that in reality
they haze together the notion of message transfer with the
notions of
application composition, which we so cleanly separate in
SCA. I don't
see
the need to force the wire-protocol composition features
onto SCA - SCA
is
already far richer in that space. SCA can use the wire
protocol features
when they are useful, but only as a reflection of the
composition goals
that
are already shown in the SCA assemblies.
>
> -Eric.
>
>
>
>
>
> Yours, Mike.
>
> Strategist - Emerging Technologies, SCA & SDO.
> Co Chair OASIS SCA Assembly TC.
> IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN,
Great Britain.
> Phone & FAX: +44-1962-818014 Mobile:
+44-7802-467431
> Email:
>
> Eric Johnson <>
wrote on 01/11/2010 20:57:15:
>
> > [image removed]
> >
> > Re: [sca-assembly] NEW ISSUE: Need some notion of
"callback"
address
> > in conjunction with eventing
> >
> > Eric Johnson
> >
> > to:
> >
> > ashok.malhotra
> >
> > 01/11/2010 21:38
> >
> > Cc:
> >
> > Martin Chapman, Anish Karmarkar, sca-assembly
> >
> > Hi Ashok,
> >
> > Just about every "broadcasting" eventing system that
I can think of
> > supports some notion of "reply-to"
> >
> > JMS: JMSReplyTo
> > SOAP WS: WS-Addressing ReplyTo (I admit this is a
little bit
of a stretch)
> > AMQP: reply-to (I think - I'm not very familiar with
this)
> > UDP: source port number
> > PGM: (piggy-backing on UDP, it looks like)
> > TIBCO's Rendevous: (yes - don't know the details)
> > Email: "Reply-To"
> >
> > ... : ??? (anyone know anything about Apple's
Bonjour?)
> >
> > Perhaps we could come up with a more complete list
of this "broadcast"
> > mechanisms, and what they support?
> >
> > In any case, the notion of using services and
references doesn't
even
> > come close to mapping on to various cases that you
might relate
to
> > eventing. Services and references are tightly tied
to the
> > interoperability and known semantics of WSDL 1.1,
whereas eventing
might
> > use a whole bunch of messaging patterns that you
might not even
be able
> > to define in WSDL:
> >
> > * message out, maybe message back
> > * message out, N messages back
> > * message out *, message back
> > * message out, message back to different component
> >
> > All of these might take advantage of a "reply to"
capability
that has
> > little or nothing to do with the request/reply
semantics associated
with
> > services and references. Also, the message back
might effectively
be
> > "any" - there might not be a specific "response"
message with a single
> > specific type.
> >
> > This question does bring me back to one question
that I've been
raising
> > for a while. What bindings do we anticipate people
using
with SCA
> > Eventing, and how do we think they're going to be
used? Absent
that
> > information, it strikes me as very difficult to
address questions
like
> > this one.
> >
> > As to whether we open this issue, since all the
existing transports
I
> > mentioned above support the functionality, I think
closing it
with no
> > action would be premature. Yet, I agree that we may
need much
discussion
> > to establish what this "reply to" notion means, and
maybe we won't get
> > there.
> >
> > -Eric.
> >
Unless stated otherwise
above:
IBM United Kingdom Limited - Registered in England and Wales
with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth,
Hampshire PO6
3AU
Unless stated otherwise
above:
IBM United Kingdom Limited - Registered in England and Wales
with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth,
Hampshire PO6
3AU
