RE: [ebxml-msg] Semantics of AS4 Receipt

Comments inline.

 

Pim

 

From: Moberg Dale [mailto:] 

Sent: 31 January 2012 16:51
To: Pim van der Eijk; 

Subject: RE: [ebxml-msg] Semantics of 
AS4 Receipt

Hi 
Pim

 

I 
have been responding to what I thought had been some questions about the 
semantics of NRR or of simple receipt. That does not involve product feature 
details. Also there are differences in what can be done when an ack of any sort 
is returned on a separate connection than on the same 
connection.

 

If 
you need to say something about delivery, then the original sender’s message has 
been delivered when the recipient has received it. If you want to say something 
about what subsequent operations of the delivered message have completed, then 
that is more a question of disposition, which for AS4 purposes need to be 
semantically aligned with the concept of “processed”   

 

The ebMS 3.0 Core 
specification makes a difference between RM-Deliver and MSH-Deliver and between 
receiving a message and delivering it.  I'm saying that I think AS4 Receipts should be 
MSH-Deliver rather than RM-Deliver, and that MSH-Deliver is closer to (without 
perhaps going as far as) EDIINT concepts. Please have a look at the 
definitions of RM-Deliver and MSH-Deliver and explain if you think an AS4 
receipt should signal RM-Deliver or 
MSH-Deliver.

 

In essence, MSH-Deliver seems to require that 
only the first of any subsequent steps (the step of getting the message and its 
metadata out of the MSH) succeeds.  It does not say anything about further 
processing steps.  But unlike RM-Deliver it seems to rule out a situation 
where an MSH sends an acknowledgment,  but then fails in some 
other SOAP module above the reliability layer and/or subsequently 
fails to transfer responsibility to the first component outside the 
MSH.

 

As 
I mentioned, it is difficult to sum up what to say about that next step after 
processed. There may not actually be a next step (because the data had been 
placed on a file system during processing, and the next step would have been to 
place the data on the filesystem. Since that has been done, it is not necessary 
to do it again.)  

 

Exactly,  writing the 
message to a file system is a way of transferring a message out of the MSH, 
if that's the selected configuration.   Whether any other application 
or user actually does anything with that message is not its responsibility and 
perhaps even invisible to the MSH. But if 
the disk was full, or the partition not mounted, or some other error happened, 
MSH-Delivery has failed though RM-Deliver has succeeded.  The MSH can 
detect this failure. 

 

 Or queues. Or another transfer over 
the same or different protocols, and so on.  

 

For a messaging 
system,  the deliver() operation  should  probably be defined as 
submit() and not as send().    But we dont't need to, and shouldn't define what 
delivery means for all sorts of integration backends.  AS4 does not specify 
what integration backends to 
use.

 

Now just how many of these kinds of next steps’ 
statuses do you want to check? 

 

I'd say we're only interested in the first step 
from inside to outside the 
MSH.

 

Your 
little pseudo code snippet abstracts over a lot of very different cases. Many 
involve delays not accompanied by a timeout value. Those would pose 
problems. 

 

We can impose or recommend that the deliver() 
operation has a timeout value if you want.   The most commonly used 
delivery mechanisms used by B2B gateway products (write to disk, JMS or other 
message queues, HTTP posts, FTP) support timeouts.  The MSH can itself 
wrap an arbitrary delivery operation in a timeout and decide how a timeout 
is to be interpreted.  But that would be out of scope for the AS4 
protocol. 

  

Retry, 
resend, resubmit options are available. Resend means that while the message had 
been sent (as indicated by a 2xx status for HTTP if that is involved), no 
expected ack (NRR or simple MDN) has been received. If the ack was lost, 
“resending” (which viewed from the sending perspective is a re-deliver) will 
result in generating another ack, as I think it should.  

 

The point I was making was that 
in some 
products,  the receiving MSH 
(not the sending MSH as in your description) has a (re)deliver 
option.  This does not require the message to be 
retried/resent/resubmitted by/to the sending MSH.  It is an internal 
function in the MSH that allows previously received messages to be delivered 
again (e.g. writing once more to a queue, or to file system, or ..).  An 
AS4 product that associates a Receipt with receive() rather than deliver() 
would not generate a new receipt. I mentioned one product that does 
generate a new Acknowledgment for redelivered ebMS 2.0 
messages.  

 

Backend 
failure is possible. NRR does not intend to magically solve difficulties by 
removing the possibility of backend failures. It does demarcate a kind of 
“responsibility” to some degree.  

 

The distinction between 
RM-Deliver and MSH-Delivery is a subtle one and indeed relates to where the 
boundary of responsibility is placed.  But neither RM-Deliver not 
MSH-Deliver depend on what happens within backends or enterprise business 
processes.

 

Customizations 
or implementation choices  of product are quite another matter. I had 
thought the question you were raising was what the standard means or 
specifies.  

 

You are right that we are 
specifying a standard protocol, not implementations. The questions are 
about how the Receive(), Acknowledge() and Deliver() 
operations that are defined in the ebMS and Web Services 
specifications relate to each other. 
 

 

 

From: Pim van der Eijk 
[mailto:]  
Sent: Tuesday, January 31, 2012 
12:49 AM
To: Moberg Dale; 

Subject: RE: [ebxml-msg] Semantics of 
AS4 Receipt

 

Dale,

 

"For example, the 
payload might be placed on a queue, but whether the payload was taken off the 
queue is an open question as far as the signal is concerned.)"

 

As I am 
writing for the third time now,  I am just asking for the success of the 
"enqueue" operation to be checked.  This does not require any new 
coordination protocols,  it just imposes some ordering in the structure 
of your application code.  Instead of something 
like

 

.... 
acknowledge(message)

.... 
deliver(message)

 

You 
would write something like

 

.... if 
deliver(message) == Success:

.... .... acknowledge(message)

.... else:

.... .... errormessage(message,'Delivery 
Failure')

 

In at 
least one Axway product, ebMS 2.0 acknowledgments seem to be triggered by 
delivery.  I know this because if a system administrator manually 
re-delivers a message from this product (which is an option that is useful 
in some exceptional situations), an acknowledgment is sent to the trading 
partner).    So assuming the AS4 implementation will follow 
this pattern,  your company's product already does what I'm asking 
for.

 

Pim

 

From: Moberg Dale 
[mailto:] 
Sent: 30 January 2012 22:49
To: 
Pim van der Eijk; 
Subject: RE: 
[ebxml-msg] Semantics of AS4 Receipt

I 
think that AS4 NRR is now roughly  the same as AS1-3 NRR. My concern was 
that we not proceed beyond secured, acknowledgement of receipt (and, when 
security features are OK, an indication that the next processing step was 
engaged). I am uncertain about “successful transfer” – the MSH did its step in 
“processing,” but whether the next processing step was successful is not 
assured. ([Many requests from “end user communities” have emerged over time to 
add on various assurances about further processing. I don’t think we should go 
there with data exchange protocols. That does not prevent embedding data 
exchange protocols in richer coordination protocols that might add on various 
notification services. But I would not want to get into these additions within 
AS4. I think the additional protocols should be developed as applications of 
existing IETF protocols, maybe as an EDIINT-NG or B2B-NG interest group. I don’t 
think it should be an OASIS effort. Just my opinion (or using the new catch 
phrase), just sayin’]

 

 

From: Pim van der Eijk 
[mailto:] 
Sent: Monday, January 30, 2012 1:36 
PM
To: Moberg Dale; 
Subject: 
RE: [ebxml-msg] Semantics of AS4 Receipt

 

Thanks 
for the response and background information.   The proposal I made was 
quite modest: to link the sending of an AS4 receipt to the ebMS 3.0 concept of 
"MSH-Delivery".  If ebMS 3.0 and WS-Reliability optionally support 
this concept, it would be inconsistent for AS4 to commit to less.  

 

AS2 (http://www.ietf.org/rfc/rfc4130.txt), 
section 7.1, relates an MDN to delivery (my emphasis):

 

"The 
signed MDN, when received by the sender of the EDI Interchange, can be used 
by the sender [... as] an acknowledgement that the EDI Interchange sent 
was delivered and 
acknowledged by the receiving trading partner [..]".

 

MSH-Delivery 
seems in fact to correspond exactly to what you call successful transfer 
to the first "receiving 
agent" that 
is indeed often "just the first step 
of a pipeline".  
The defining requirement is that the message has been transferred 
successfully out of the AS4 MSH. This is the case for all 
the examples in my message. The increase in latency is minimal, the 
impact on implementations limited, and fewer false receipts are produced. 
The functionality better matches the expectations of end-users and is 
supported in commonly used B2B gateway products for other 
protocols.

 

Pim

 

From: 
 [mailto:] On 
Behalf Of Moberg Dale
Sent: 30 January 2012 17:53
To: 
Pim van der Eijk; 
Subject: RE: 
[ebxml-msg] Semantics of AS4 Receipt

The 
first observation is that the requirements internet-draft was never published as 
an informational RFC.

If 
it had been, some of the tradeoffs involved in being an “applicability 
statement” and in seeking to make AS1, AS2 and

AS3 
as much alike as possible would have necessitated some revisions to the original 
draft. The original draft also included several ideas that were not 
implemented.

 

Going 
back over all the tradeoffs would be tedious. I think the main one relevant here 
is that the MDN was reused as the message type for 
receipts.

 

In 
an email context, a MDN is more associated with what the a UA does upon seeing 
what the final MTA has come up with. So POP3 and IMAP based UA access can also 
generate MDNs. EDIINT, however, only makes use of a success code from the MDNs 
of “Processed”. Here is where the semantics that got implemented got 
defined.

 

RFC 
2298 was I think in scope when the EDIINT work started, but I don’t think all 
the subsequent “obsoleting or updating” RFCs in this area change the 
situation.

 

"processed"    
The message has been processed in some manner (i.e., by some sort of rules or 
server) without being displayed to the user.  The user may or may not see 
the message later, or there may not even be a human user associated with the 
mailbox.

 

The 
“processed” code then indicates the semantics of any kind of 
receipt/delivery/disposition on the receiver side. So the overly ambitious 
requirement of the draft were aligned with the constraints of reuse, and what 
already existed. The reasons for the tradeoff were many and varied. Mainly 
however they all reflected the unpredictable latencies between 
MTA/MSH/WebServer/FTPserver reading of data and the acquisition by the next 
processing step(s). Would it be a POP3 UA or was there an automated pipeline 
pushing to a UA feeding a translator,JMS, MFT, ESB, or any of the integration 
technologies in use? Or for HTTP, would a servlet, appserver, CGI push to the 
next app, and would a success from that app be relevant to count as 
“disposition”. The ultimate “backend  application” is only in simple 
middleware cases just one hop from reception; the receiving agent is just the 
first step of a pipeline (where there can be  branching or forking process 
flows). How is all that to be summed up in one code?

 

In 
general the A2A visibility problem remains unsolved for general deployment 
situations by any current standard. It will take some significant effort and 
cost to make A2A visibility commonplace. AS4 will not be doing it IMO. The 
backflow of status notifications down the integration pipeline is just too 
varied to smooth over, without considerable thought given to lightweight 
notification technologies, with built in subscription, expiration, digest, and 
other capabilities. 

 

 

 

 

Hello,

 

I was working on an 
AS4-related topic over the weekend and have some questions on AS4 non 
repudiation receipts.

 

The AS4 profile is 
presented as designed to meet EDIINT requirements.  

 

According to section 
3.7 of http://tools.ietf.org/id/draft-ietf-ediint-req-09.txt EDIINT 
defines the requirements for non repudiation 
as:

 

"Receipt by the 
sender of a signed receipt, is an implicit
   acknowledgment of 
successful mailbox delivery. It is also an
   explicit 
acknowledgment that the Interchange was retrieved from
   the 
mailbox - pick-up notification. By having the receiver sign
   the 
receipt, it authenticates that the intended receiver picked up
   
the EDI Interchange -- mailbox authentication -- and that the
   
intended receiver verified the integrity of the EDI Interchange,
   
and the identity of the sender. By returning the original 
message
   id and the one-way hash value of the received contents 
back in the
   signed receipt, the sender can reconcile the 
acknowledged EDI
   Interchange with what was 
sent."

 

According to section 
8 of http://docs.oasis-open.org/ebxml-msg/ebms/v3.0/core/ebms_core-3.0-spec.odt the 
ebMS 3.0 Core Specification distinguishes between RM-Delivery and 
MSH-Delivery. 

Case 2: The acknowledgment is "on MSH-delivery" 
(supported in WS-Reliability). In that case, notification option 1 can be 
supported as well as option 2. In order for option 1 to be supported, an RMP 
must implement RM-Deliver operation so that it is only considered successful 
(worthy of sending an acknowledgment) if the Deliver operation from MSH to 
Consumer also succeeds. It is RECOMMENDED that an implementation support this 
acknowledgment semantics.

In http://www.oasis-open.org/apps/org/workgroup/ebxml-msg/download.php/44888/AS4-profile-v1.0-wd22.odt 
section 3.4, the following is stated about AS4:

 

"The Receiving MSH is 
taking responsibility for processing this user message. However, no guarantee 
can be made that this user message will be ultimately delivered to its Consumer 
application (this responsibility lays however now on the Receiver 
side)."

 

Now this seems to be 
more like the RM-Delivery (only option supported in WS-ReliableMessaging) than 
MSH-Delivery ?   

 

The EDIINT definition 
of "non repudiation of receipt" seems to require MSH-Delivery rather 
than RM-Delivery.  If we're claiming that AS4 meets EDIINT requirements, is 
this enough ?

 

I'm not saying that 
AS4 products should support a VAN mailbox interface based on pickup of a 
message, and should delay sending a receipt until such a pick-up happened 
(possibly hours or days later).   Drawbacks of this have been 
discussed in the past when we looked at multi-hop reliability models. 

 

But in the AS4 
context, it could be defined as:  the MSH successfully passed 
responsibility for delivering the message to some third entity (system, program, 
user).

 

Examples of succesful 
or unsuccessful delivery could depend on 
implementation:

-   
Delivery to a mailbox:  

    
Succes:  message was added to mailbox.  

    
Failure:  quota exceeded, mailbox full.

    
(No guarantee that any user or system actually processes the 
message)

-   
Delivery to a file system:    

    
Success:  the MSH successfully created, wrote data to, and closed 
a file representation of the message metadata and contents.  

    
Failure:  Permission denied,  I/O errors

    
(There is no guarantee that any user or program actually processes the created 
file)

-   
Delivery to a message queing system

    
Success:  successful enqueue()

    
Failure:  enqueuing failed 

    
(There is no guarentee that any other application dequeue the 
item)

-   
Delivery to another B2B messaging system:

    
Success:  the Submit() operation was invoked 
successfully

    
Failure:  Submit() operation failed

    
(No dependency on any downstream receipts)

 

In the case of 
failure, which the AS4 MSH can detect immediately after receiving a 
message,  I would prefer the AS4 to not send a Receipt but to send an Error 
instead. 

 

An alternative idea 
is discussed in section 8.2.4 of the ebMS 3.0 Core Specification, which has 
a discussion on false delivery failures:

 

"False DF - which can 
never be completely eliminated - can always be detected outside the reliable 
messaging (RM) layer, in a tractable and less urgent way - e.g. the sending 
party may synchronize on a daily basis by communicating its list of assumed 
delivery failures, for confirmation by receiver. The Status Request feature 
(which may be described in a forthcoming Part 2 of the ebMS specification) could 
facilitate this function."

 

Pim