Not true. There is no need to have the same
Service/Action values on all user message units in the bundle. The submit()
and deliver() operations of the interface to MSH are still per-unit, so
message units (that don't have errors, including the EBMS:0031 error)
will be delivered individually, with metadata that enables routing to the right
application.
In a multihop environment, the primary message unit is used
for routing, so if you want to bundle invoices with POs (and configure
Pmode[].bundling.compatibility.pmodelist accordingly), you
should make sure that the ultimate receiver MSH can deliver to both the
invoicing business application and the ordering business
application.
Pim
From: Moberg Dale [mailto:]
Sent: 02 October 2009 00:11
To:
Subject: [ebxml-msg] Another problem
with the RootConvention (one P-mode) WAS [ebxml-msg] Bundling concerns,
continued
Besides making bundling
non-modular, the RootConvention appears to prevent the normal use of metadata
for purposes such as “application level routing” – that is the routing of a
payload of a message to an internal application or queue as part of the
integration orchestration.
If the root P-mode is
used, all the User Messages must use the root P-mode Service, Action in its
CollaborationInfo section within UserMessage. But if Service and Action have
been used for routing, then all UserMessage parts will have the same metadata
values. So if we have Purchase Orders and Invoices in the same bundle, they all
get routed to the same place in violation of business rules. When the messages
are sent individually with their correct P-mode metadata values, then they can
be routed correctly.
Since ebXML leaves it
wide open which combinations of metadata are used for internal purposes of
routing, it becomes difficult to see how bundling with the RootConvention will
allow the correct dispatch to occur.
From:
Moberg Dale
[mailto:]
Sent: Thursday, October 01, 2009 8:54
AM
To: Pim van der Eijk;
Subject: RE: [ebxml-msg] Bundling concerns,
continued.
The different
processing in the bundling case is contrasted with individual message:
When the message is not bundled, its P-mode as determined by its
user message metadata is used. In bundling, the processing ignores how the
message would be processed by its P-mode retrieved by its metadata, but must
instead apply a different retrieval method for P-mode, and different processing.
It makes a difference in what is done whether it is in a bundle or not. Even the
presence of metadata that would govern processing in one case, has to be ignored
when bundling. That violates Modularity, which I would like to see work. So I am
not a fan of the RootConvention at all, even in its current lexical ordering
version.
MSHDiscretion means to
me that bundling is just supposed to be a transmission option, and if it were
that, then a fallback to an alternative transmission mode would work, just as in
ebMS 2 when two delivery channels were accepted for a given action. So that
motivates trying to see how a fallback would work. It doesn’t. So I agree that
we should not really think of bundling as a transmission option. If it fails, it
is logged and manual resubmission is probably
needed.
I think it is still a
very complex proposal. I will study the sections you mention before the next
meeting to see whether they simplify but I am hoping we can reduce complexity
and not add features and checks and processing overhead.
From: Pim van
der Eijk [mailto:]
Sent: Thursday, October 01, 2009 8:32
AM
To: Moberg Dale;
Subject: RE: [ebxml-msg] Bundling concerns,
continued.
Comments
inline.
Pim
From:
Moberg Dale
[mailto:]
Sent: 01 October 2009 01:15
To:
Subject: [ebxml-msg] Bundling concerns,
continued.
There are several design principles
and conventions at work in our current approach to bundling that don’t quite
hang together cleanly -- or so it seems to me. Let me describe some of the major
ideas and then discuss some of the tensions in getting them to work
together.
Modularity: Each user
message should behave in a bundle just as it would by being sent on its
own.
Partly true. For
routing, transport and security (except receipts) and reliability the secondary
units are being processed like the primary unit.
MSHDiscretion: A MSH
should be free to be statically or dynamically configured to send messages in a
bundle or individually.
RootConvention: The P-Mode
of the SOAP body user message (href attribute omitted in PartInfo) is
primary.
No, the primary message
is the first eb3:UserMessage in the eb3:Messaging container, which is a SOAP
header. It doesn't matter where the payloads of that primary message are stored
(in the SOAP Body, as an attachment). And perhaps the eb3:UserMessage
doesn't even have payloads, as the minOccurs of eb3:Payloadinfo is
0.
SingleSignature: When a
signed message is indicated by the P-Mode of the Root the signature must cover
all user messages.
The PMode of the
primary unit specifies what (if anything) is signed. If it says: sign the
eb3:Messaging unit, the SOAP Body, any other SOAP headers per WS-I RSP, and all
payloads, then the effect is that all user messages are signed.
OneSharedReliabilityId (sequence
number in WS-RX)
Tensions in
Modularity Root Convention, and Single
Signature
Suppose one non-root body part has a
P-mode with enveloped encryption. The assembly of parts is to proceed as it
would for individually sent messages even when in a bundle. But the P-mode of
the root is signed only. Should the non-root part be encrypted and then signed?
Or should encryption be omitted and the receiving end indicate a
Value-Inconsistent error? See section
3.2.2 of the latest draft. Whether or not to encrypt payloads is
determined by the primary unit's PMode, since this is specified in
the Pmode[1].Security
PMode section. This is
true for both sending and receiving MSH, so there is no Value Inconsistency
error. Or when bundled should each non-root
part be processed by ignoring the defined P-mode for this user
message?
On the face of it Modularity can
easily conflict with RootConvention when SingleSignature is operative. In fact
whenever the root P-mode conflicts with a P-mode of a non-root user-message
part, then the normal message receiver-side error checks must be disengaged.
Likewise I think during sender message assembly, the access to the P-mode
associated with the user message’s AgreementRef will need to be overridden. This
is a lot of special coding and almost amounts to a new mode of operation. Bundling is functionality for the
ebMS module, the security and reliability modules don't need to know much if
anything about it. Suppose you have an existing ebMS3 implementation that
you want to adapt to support bundling. The only thing is to check that it
picks the right (=first) UserMessage for PMode setting (agreement)
validation, since the only new thing is that there may be more of them. The
security and reliability modules just use a different key to the agreement/pmode
store than the ebMS module. Please explain why this would be a lot of
special coding.
So I agree with Sander that it might
be better to restrict user message bundling to messages with the “same” P-mode,
assuming we can explain what that means. I doubt that this constraint will be
sufficient to solve all the tensions, however.
If trading partners
want to restrict bundling this way, then the parameters in section 3.3 give them
the configuration parameters they need to do so. But if partner P1
exchanges 30 types of messages with a trading partner P2 (hence 30 PModes), it
may be perfectly acceptable to use a single set of security/reliability
parameters for all 30 message types and to allow all of them to be bundled
together, and this will increase the benefit on bundling. Many current
ebMS user communities use configurations based on a handful of profiles (or CPA
templates). If they exchange messages regularly, they immediately
benefit from bundling.
Tensions in
MSHDiscretion and OneSharedReliabilityId
Here is a case I was struggling with
describing today. Supposedly the MSH can “just decide” to bundle messages
together. And if the bundle does not work for some reason (errors come back
about message parts) it could decide to “back off” and send messages separately.
In the case of a signature failure, I think we agree that it might be reasonable
to see whether each part could be sent individually. Presumably this makes sense
because no part “should” be delivered, given that we have SingleSignature.
This is certainly an
option, but in practice I suspect some system manager will want to inspect this
message and see what went wrong, instead of having a policy to automatically
sending the units unbundled. We could define a PMode parameter to do this,
or leave it as an implementation choice.
If the error is due to
e.g. the use of an untrusted certificate, then the primary unit will also be
rejected when sent unbundled. There could be a benefit in sending a
secondary unit whose PMode does not specify signing (or references another
-trusted- certificate) unbundled, since they would not suffer from the
certificate errors of the primary unit's PMode. But this doesn't help if,
as you and Sander want, only units that have the same PMode are bundled:
then you are certain that they will all fail when sent unbundled, since
the PMode specifies the certificate and it is the same for all. So then, when a
bundle fails, all units in it
fail permanently.
If there is an error in
calculating the hash for some cid: MIME attachments and not for other, this
looks like a product error that would probably also occur with standalone units.
Resending the units is unlikely to solve anything .
Sending each part individually would at least be
preferable to sending the same bundle again. This is because the message
processing failed before the bundle was acked, and so retrying the same bundle
with its security defect could go into an endless loop of retries as long as
acks might not be sent.
Yes, just as you
presumably do not resend a failed unbundled message unit.
Now suppose for a different bundle
that its security checks out, and reliability is processed, and we arrive at the
separate message parts. Resending the whole bundle would probably just be acked
as a duplicate, so that will not be an option for a transparent recovery. So if
we do get error messages, should we trust that the parts for which we did not
receive an error message were ok and delivered? What happens when you send an unbundled user message unit,
you get an RM acknowledgement and no error. Why would you not question
whether it was delivered in that case, whereas you have doubts about it being
delivered if it is sent in a bundle? You can specify that you want an NRR
receipt. Those are specified per message unit, so you would get
separate eb:Receipts for each user message unit that didn't fail.
What if (some of ) the error messages get lost? What if an error for an unbundled message
is lost? With WS-ReliableMessaging, an acknowledgment does not mean that
no error occurred, even when using InOrder delivery. There is no difference when
bundling applies, so why is this raised as an issue with bundling? Not
sure how to specify it in PModes, but an MSH could send errors back over a
reliable sequence, to make sure they don't get lost.Or to be
on the safe side should we send all the parts again, individually, adhering to
MSHDiscretion and the hope that a more determinate outcome of errors and/or acks
comes back? If our received error messages were infallible indicators of exactly
what went wrong, then we might try to send each payload with an error and see
whether that remedies the error message. Otherwise, we retry all the body parts
separately. The problem here is that if some of the parts had been ok and had
been delivered optimistically, then retrying individually will get duplicates
delivered. Yes, that's why I think
resending a message unit that you did not get an error for separately,
if you sent them before in a bundle that you got an acknowledgment for, is a
questionable idea. You have to build a second level of duplicate
elimination at the ebMS level that you otherwise wouldn't need.
And that violates our modularity assumption because when each part
has its own reliability id value, duplicates would be eliminated. It could be
that we just give up and do nothing about these ebMS level errors, just logging
them. I think the receiving
MSH should generate errors for the units that have errors and no errors (or
receipts, if NRR is requested) for the units that are fine at
ebMS level. We could mandata the errors for units are always bundled, so
that the sending MSH receives them together, providing strong evidence that some
units failed but others not (since that bundle would contain explicit errors for
those if they were). But that would violate
modularity too, because in that case, we would not know that any of our parts
got delivered successfully as we could with an acked, errorless single message
sent individually. Not
true, with WS-ReliableMessaging, an acknowledgment does not mean that no
error occurred, even when using InOrder delivery, also with unbundled messages.
Being part of the bundle has made our state alignment a bit
foggier in the case of some errors at the ebMS header level. Or so it seems to
me. Have a look at 3.4
and 3.9.3.
Let me stop with these two issues,
and ask for simplifications that would diminish the interactions threatening
Modularity. I think that Sander had a good suggestion to disallow P-modes that
were “different” but I would like a cleaner way of stating this requirement. I
personally would not mind dropping the single signature idea because that
definitely makes sending a message in a bundle different from sending it on its
own. I think that if signatures
are per unit, can use different certificates or algorithms, that some units use
encryption and others not we are loosing the biggest efficiency benefit for
bundling. In that case, I would not use bundling at all. Or define a
batching protocol (multiple SOAP envelopes in distinct MIME parts)
if saving a separate HTTP connection set-up is worth the effort.
With batching the various SOAP envelopes could also have separate reliabilty
sequence numbers.
(One security header can have many
separate signatures within it, although I need to check WSI BSP to see that they
have not profiled that away.)
As far as ebMS level errors go,
either we should state that nothing is to be done in response to an error by the
sender as part of our specified protocol. Or we might state that each part is
sent again individually (but I think this is bad because the receiver will have
to support a second layer of duplicate detection by ebMS message id). Or we
state that a receiver MAY retry a message with an error received when it was
sent in a bundle. My preference is now headed toward saying that the specified
protocol does not say what to do when an ebMS error is received for one part of
a bundled message. Maybe we even say the same thing about the failed-signature
error message when received by the sender. Please review 3.4 and
3.9.3.
I do not think that going into the
dependency graph direction is going to make this situation simpler or better. I
think that we should say that when choosing bundling, that is the mode of
transmission, and when it fails, each message payload must be resubmitted by
operations (but with a do-not-bundle flag set! ) Please review 3.4 and
3.9.3.
