So this proposal is still geared to bundling within the
same PMode , primarily, which I think is good and safe.
I like the optional list of PModes for controlling a
broader bundling.
I think we should discuss it.
Jacques
From: Pim van der Eijk
[mailto:]
Sent: Wednesday, July 01, 2009 10:58
AM
To: Jacques R. Durand;
Subject: More on message
bundling
The latest Part 2 draft already has a good start on
bundling.
Here are some more
ideas, building on that.
A relatively
simple way of controlling bundling is to view bundles as combinations of a
single "primary" user message (the driver) and other "secondary" user messages
that piggy back onto that message. The "primary"
user message determines the P-mode settings applied by the MSH to the entire bundle. Assumption: in a bundle, the primary user message is
the first message unit and the first message unit can be assumed to be the
primary user message.
Since user messages are
submitted separately and usually not at the exact same time, the MSH should
temporarily store submitted user messages to allow combining them later into
bundles. The MSH may already compute the eb3:UserMessage XML header
including the eb3:PayloadInfo references to message parts. It may also already
perform some CPU intensive ebMS3/AS4 functions such as compressing payloads and
perhaps already compute secure hashes, as an optimization to speed up security
operations in the security module. The MSH cannot yet assemble the ebMS3 SOAP
header and does not perform any security operations such as encryption, since
that depends on the Pmode of the primary message that a user message is going to
be bundled with, nor does it apply any
reliability headers for the same reason.
Bundling therefore would be controlled controlled by Pmodes.
Suggested new Pmode parameters:
Pmode[].bundling.standalone: values "always", "optional",
"never"
Indicates whether
the user message is to be sent as a standalone message without piggy-backed user
messages
("always"), may be sent separately but may also be bundled
with other user messages ("optional") or is never sent as a standalone
message ("never"). An ebMS 3 implementation that strictly conforms to part 1 of the ebMS 3.0 Core
Specification and does not support any form of bundling will only support the
option "always" for this parameter. A parameter value "never" means that the
message needs to be bundled with some other message to be sent at all. It cannot act as a primary user message.
Pmode[].bundling.bundleswith
This parameter is to be
specified in case the "standalone" parameter has a value other than "never". Its
value is a list of Pmodes. The semantics of the parameter is that a user
message may (if standalone is "optional") or must (if standalone is "never") be
bundled with other user messages associated with the Pmodes identified in the
list. The parameter may list other types of Pmodes that describe messages
that the user message can be bundled with. In general these Pmodes SHOULD
be destined to the same target MSH (even in a multihop network) and SHOULD
satisfy the same or stricter QoS
requirements, but the responsibility for making sure this is true is with the
Pmode configuration administrator. Typically, the list value of the
"bundleswith" parameter for Pmode P1 will also include that same Pmode P1,
indicating that more than one user message of this type may be sent. But
if a particular user message is known to be very large or to be sent rarely, the
parameter could only list some Pmodes of messages that are known to be small and
sent frequently.
The MSH should support
configuration of the maximum duration of the time it waits for more user
messages to be submitted to be added to bundles, the maximum number of user
messages it bundles and/or the maximum size of the user message content
(including business documents and attachments, after AS4 compression) of the
bundle.
Pmode[].bundling.maxdelay
Pmode[].bundling.maxmessages
Pmode[].bundling.maxsize
Note that the MSH
is not required to wait before the maxdelay interval has expired before sending
the message. It may send the message before the interval has finished. The maxdelay is a contract with the submitting
business application which may have some time-to-perform to be respected.
If the "bundleswith"
list includes multiple Pmodes, and more than one candidate waiting "primary"
user message is available that a user message can be bundled with, the
allocation is implementation-dependent.
If the value of
the "standalone" parameter is "optional", the decision by the MSH to send the
message as a standalone message or to bundle it with another user message is
implementation-dependent.
If the value
of the "standalone" is "never", and no primary message is available to bundle
the message with within a "maxdelay" duration, the MSH MUST generate an
ebms:xxxx BundlingError sender error.
Bundling and
Pull
The description
above assumes that submitted user messages are queued temporarily between ebMS
processing and between further security and reliability processing. This
means that a submitted user message may
not yet be available for
pulling, leading to unintuitive
results. This could be optimized in
advanced implementations as some kind of on-demand bundling, or
on-demand MSH completion, triggered by an incoming Pull Request.
Left to implementations.
Bundling and
Sync
The above is really about requests and responses for
both Push and Pull binding, but not for response user messages to Two Way / Sync
MEP bindings, which must be transmitted on the HTTP back channel without
delay. The synchronous responses to
each of the user messages in the bundle should be sent without waiting
"maxdelay" for more messages to be bundled with them, to prevent HTTP transport
timeouts.
Minor comment: the
current bundling chapter writes:
"However, when InOrder
delivery assurance is required for User message units, bundling SHOULD NOT be in
effect for these messages as there is no trace of the submission order on the
sending side, and the order of message units under eb:Messaging is not
significant."
I think we could
assume that message units are to be
processed in document order. There is still no guarantee that
messages are assigned to the same bundle and/or that different bundles would
arrive in-order though, so the warning against
bundling is valid. Perhaps an advanced MSH API could allow bundling or
ordering to be controlled. Out of scope for
spec.
Bundling and multihop
Assumption:
intermediaries (such as the "transparent" intermediaries) may not bundle or
unbundle messages. The MSH may not know which Pmodes target the same (logical)
destination (as this requires knowledge of the routing function). So what can
and cannot be bundled cannot be predicted based on Pmode parameters other than the proposed "bundleswith". It seems acceptable to assume that in a
multi-hop context any routing function should be able to look at the first user
message in a bundle to determine the destination.
So the "first unit determines .." principle would apply to both bundling and
routing.
Pim
From: Jacques R. Durand
[mailto:]
Sent: 08 April 2009
04:18
To:
Subject:
[ebxml-msg] follow-up on message bundling
Follow-up on bundling:
Here is a short list of issues to be discussed before
we start drafting anything on bundling:
1. Agree on the packaging of
bundles:
- several UserMessages units in same
eb:Messaging?
- what other SignalMessage unit(s) can go with
it?
2. What is expected to be common to all messages in the
same bundle? Here is a list, by decreasing "obviousness":
- security and reliability.
- the destination. (However: in case of multihop: could
a bundle be broken down by the last intermediary, and forwarded to
multiple final destinations?)
- other QoS parameters? (how errors are supposed to be
reported,
Receipts...)
- the MEP? (could some message units belong to one-way
and some to two-way? do we expect two-way responses to be bundled too?
what if both parties do not have same bundling
capability?)
- the MPC?
Should there be some "profiles" of bundles depending on
what is common to their messages?
3. Can we pull a bundle?
- should the PullRequests themselves be bundled to
indicate intent and ability to get bundles? SHould PullRequest signal be
extended? Or could a single, normal PullRequest get a bundled (PMode
config)?
-jacques
From: Jacques R. Durand
[mailto:]
Sent: Wednesday, March 11, 2009 4:29
PM
To:
Subject: [ebxml-msg]
an outline of message bundling
Definition:
-------------
Message bundling / chuncking:
(a) ability to group several User Message units in a
single ebMS envelope, representing
several logical message
units to be delivered separately on
reception.
(b) ability to split a large payload over several ebMS
envelopes and SOAP messages,
and to deliver by Receiving MSH side as a single
unit.
Reminder of bundling cases
defined in Core Spec:
-----------------------------------------------------------------------
The following is a non-exhaustive list of valid bundling cases:
(a) eb:Messaging element with the following children:
an eb:UserMessage element
an eb:SignalMessage element with an eb:PullRequest child
(b) eb:Messaging element with the following children:
an eb:UserMessage element
an eb:SignalMessage element with one or more eb:Error
children
(c) eb:Messaging element with the following
children:
an eb:UserMessage element 1840
an eb:SignalMessage element with an
eb:PullRequest child
an eb:SignalMessage element (distinct from the previous
one) with one or more
eb:Error children
(d) eb:Messaging element with the following children:
an eb:SignalMessage element with an eb:PullRequest child
an eb:SignalMessage element (distinct from the previous
one) with an eb:Receipt child
Bundling of ebMS Messages
-----------------------------------------
1- Rationale
- performance,
simpler QoS processing
2- Packaging of bundles
- Bundling definition
(several UserMessages units - and SignalMessage units - in
same eb:Messaging, NOT several eb:Messaging
headers)
- processing
model
(idea: semantic equivalent to a sequence of User
Messages)
(sending side: before RM and Security, receiver side:
unbundle after Security and RM)
- bundling of
payloads
(SOAP Body, attachments)
3- Controlling bundling
- constraints
or recommendation on what message units can be bundled together.
(e.g. same MPC? same security credentials? reliability
level? MEP and channel binding?..)
- PMode
parameters
(bundling based on qty, based on time)
- Bundling of
Receipts.
- Bundling of
Errors.
4- Transfer Considerations
- The case of
pulled messages
(need to handle this: in a multihop config, a message can
be pushed then pulled)
(such bundles must all be assigned to same
MPC...)
- Routing
bundled messages
(routing info: which UserMessage unit to
use?)
Jacques
