[ebxml-iic-interop] FW: Feedback from eBES on our Interop Test suitedraft

Title: RE: IIC specs

Our 
interop sub-team may be interested in this feedback

from 
Christopher Frank , architect in the eBES interoperability pilot 
project.

(I 
answerred it partially)

 

Christopher had also a question on whether there was 
implementation work plans

for 
our test framework. We need to be able to answer such questions, even 
though

that's 
beyond the IIC charter.

 

Regards,

 

Jacques

 

-----Original Message-----
From: Jacques Durand 

Sent: Monday, October 21, 2002 4:48 PM
To: 'Frank. 
Christopher'
Cc: Jacques Durand
Subject: RE: IIC 
specs

Hi 
Christopher:

 

I'll 
forward your comments to the IIC team.

See 
inline comments.

 

We are 
(in IIC) definitely interested in keeing in touch with you and 
eBES,

and in 
getting such feedback.

 

Cheers,

 

Jacques Durand

 

  
-----Original Message-----
From: Frank. Christopher 
  [mailto:]
Sent: Thursday, October 17, 2002 2:00 
  AM
To: Jacques Durand
Subject: AW: IIC 
  specs

  
Hello Jacques,

  
 

  
Thanks for the quick respond. 

  
 

  
I 
  assume that we are able to decide on the core structure of our test spec this 
  week (I wrote a proposed draft therefore).

  
 

  
My 
  personal comment after studying the IIC specs (req, test framework, MS 
  conformance, MS interop):

  
In 
  general while reading the IIC drafts I had several issues when trying to 
  relate the spec content to real-life test implementation (therefore I asked 
  for the physical sources). 

  
One 
  of the basic targets of the test framework is to not request too much effort 
  on implmenting the test framework on a physical system (where I agree a 
  100%;-). Reading the service description and its very detailed action 
  description, I doubt that can come to truth (depending on the ebMS 
  implementation out there, of course).

  
But 
  this is may because I am nitpicking.

  
 

  
I tried to be more "tolerant" when interpreting the 
  Actions of the test service (which I see as the major physical impact), what I 
  found is this: Interpreting the Actions as "activity description" (thereby 
  ignoring several details in the Actions description), I found them very, 
  very useful for defining Test Cases. 

  
 

  
Use this as an example (where all "activity" marked 
  words are representing a activity (actually sometimes a test framework action 
  and sometimes a core ebMS functionality):

  
 

  
TestCase n_00

  
(The table is to read from left to right, top to 
  bottom and is then equal to the time-line while the Requestor and Responder 
  work independently.)

  
 

  
Step    
  Requestor                                                
  Responder

  
1        "Send" 
  ebXML20(mpld_00)                        
  

  
2        
  "TimeoutVerification"                                 "Receive" 
  ebXML20(mpld_00)

  
3                                                                       "Reflector"

  
4                                                                       "Send" 
  ebXML20(mpld_00)

  
5        "Receive" 
  ebXML20(mpld_00)                    
  "TimeoutVerification"

  
6        
  "PayloadVerify"

  
 

  
<Jacques> we could indeed use such an activity 
  diagram (or "swim line") that shows events on each 
  side,

  
if only for more intuitive description of the test 
  case (I think we should.)

  
When it comes to a more formal description, because 
  the test case is entirely controlled by the Test Driver 
  

  
(and because we only want to 
  trace it in the Test Driver only, for 
  convenience), we define it uniquely in terms 
  

  
of sequence of events that 
  occur in the test Driver (which will be, say, on the 
  Requestor side in your example).

  
What is happening on the other side, is only a 
  "reaction" to such events (e.g. an Action is 
  triggered

  
that sends back an event), so we only observe its 
  effect on the Test Driver (e.g. a messsage has been 
  received

  
or not, or is corrupted, 
  etc.). The test outcome will be decided on these observable 
  "effects".

  
 

  
those test cases then could be easily extended (or 
  adapted) by doing:

  
 

  

  
5        "Receive" 
  ebXML20(mpld_00)                    
  "TimeoutVerification"

  
6        
  "PayloadVerify"

  
7        "Send" 
  ebXML20(%error%)

  
8        
  "TimeoutVerification"                                 
  "Receive" 
  ebXML20(%error%)

  
 

  
I would actually not care (or define) "how" Requestor 
  or Responder perform the "Reflector" (as you in your current spec do not care 
  (for good reason) "how" they perform a "send" or how they "parse" a ebXML 
  message. As long as the rules and conditions of the "Reflector" 
  activity are completly desribed (of course) . Where I found the test 
  services "action" description too implementation-oriented (and I am saying 
  this as technician;-).

  
 

  
<Jacques> we 
  certainly do not intend to go further in implementation 
  details...

  
while we do not 
  specify how the Action code should be implemented, it is 
  true

  
that our description 
  is detailed about a few things, notably:

  
- the actual content 
  of the messages that the action generates.

  
This is because in the 
  test suites that use these Actions, there will be formal descriptions 
  of

  
test Conditions that 
  will make precise assumptions on the content of the message they 
  generate.

  
- notification aspects 
  (this specifies what / when the Test Driver should get 
  notified.

  
(Was that the kind of 
  details you were concerend about?)

  
 

  
Additionally (now speaking for the Interop Test spec 
  only):

  
With a few years of EDI/B2B experience, I would 
  strongly recommend to extend the Test cases (or flavors of them) to define 
  "stress" testing in regard to "a-lot-messages-at-one-time" and 
  "large-messages" (and both in combination, of 
  course;-)

  
 

  

  
<Jacques> We are 
  fully aware that this kind of scalability testing is also 
  required...

  
We will look into this 
  after we are done with the "basics" of interoperability, 
  and

  
would then welcome 
  your experience to help in this..

  
By end of this month, 
  we plan to wrap-up a more complete draft of our "basic interoperability 
  profiles", that should

  
only be considered as 
  the baseline for ebXML interoperability - but one that all users could agree 
  with, 

  
independently 
  from more specific 
  requirements they might have in addition. The idea is that more 
  advanced

  
"interoperability 
  profiles" will be added after that (each of these concretized by an interop 
  test suite - or

  
a sequence of test 
  cases) . 

  
 

  
Hope 
  that gives you an idea of my impressions and you may find it useful as input 
  to your current work.

  
 

  
I will keep you posted on the result of the discussions in the European 
  pilot.

  
<Jacques> Yes 
  please. We'd like to know what 
  are your plans and milestones for this 
  pilot.

  
 

  
Regards,

  
 

  
Jacques

  
 

  
Thanks again.

  
 

  
Best 
  Regards,

  
Christopher

  
 

  
    
-----Ursprüngliche Nachricht-----
Von: Jacques Durand 
    [mailto:]
Gesendet: Mittwoch, 16. Oktober 
    2002 01:28
An: Frank. Christopher
Cc: Jacques 
    Durand
Betreff: RE: IIC specs

    
Hi Christopher: 

    
Thank you for your attention. 
Yes, 
    some of our members are planning to implement 
a 
    reference implementation of the Test Service, as 
well as of the Test Driver. 
We are lining up 
    resources for this - while still working 
at 
    completing our test specs - as there is definitely demand 
for such tools in B2B user forums, not only in US (OAG, NIST...), 
    
but Asia (ECOM) and Europe as well. 

    
As ebXML IIC will support and advise on the 
    deployment 
of its test architecture, we are 
    naturally interested in learning 
more about your 
    specific requirements, and project plans. 
So we 
    could remain in touch. 

    
Also, we are currently working on our MS Interoperability 
    test suite, 
and should have an updated draft by end 
    of October - so please understand 
 that the 
    current draft is not stable yet. 

    
Best Regards, 

    
jacques Durand 
ebXML IIC 
    Chair 

    
[Jacques 
    Durand] -----------------------------------------------

    
 

    
The CEN eBES and OASIS started a "European Interoperability 
    Pilot Project" 
(you may are already aware of). I am 
    member of the Messaging Team currently 
putting 
    together a core spec for our pilot project. 

    
Our pilot will focus on a real-life business case 
    implementation, but we are 
trying to refer to the 
    work of the IIC (especially the ebMS interop spec) as 
much as possible. 

    
After studying the IIC specs trying to determine the 
    agreements and 
disagreements to the European pilot, 
    I was wondering, whether you have or 
plan to have 
    any physical implementations of the TestService 
(ebXML_IIC_Testing) (and its actions) as desribed in the 
    TestFramework spec? 

    
Best Regards, 
Christopher 
    Frank 
(SD Research/Protoyping) 

    
-- 
SEEBURGER AG, Edisonstrasse 1, 
    D-75015 Bretten, Germany 
Fon:+49(0)7252 96-1185 
    Fax:+49(0)7252 96-2400 <http://www.seeburger.de/>