[ebxml-iic-msg] RE: other Schema docs - usage?

Title: Welcome

Michael, Matt:

 

I 
believe the "Test requirements" document and schema are on good 
tracks.

Let me 
try to summarize where we are, and raise some remaining 
issues:

 

1- The 
Requirements schema/doc, so far captures Level 1 ("core", or most of it). 

When 
the corresponding tests are run (controlled by ANT + JXUnit), one can 
check that 

an 
ebXML message is well-formed and also semantically sound, 
according to Level 1. 

That 
actually captures "wire-conformance" Level 1.

 

2- We 
expect to drive the candidate MSH so that it generates a diverse-enough sampling 
of messages,

each 
of which will undergo the Level 1 "wire-conformance" tests 
above.

 

3- 
Issue: as we will drive the candidate MSH to generate messages to be tested, it 
is not enough

to 
test each of these through the "Level 1 tests" above. Indeed, we want to make 
sure in addition that

the 
set of generated messages (test set), uses all features expected in Level 
1.

Example: if for some reason the candidate MSH fails to 
generate some optional message elements - e.g. 

digital signature, etc. - then its message format may 
still be correct, as tested on the wire, but

we 
cannot claim Level 1 passes, as the test coverage would not have checked all 
expected message features.

Solutions: we need to have a way to "require" and to 
check, in the generated message test set,

that 
all expected message features have ben produced at least once. That could be 
done by (a) either

keeping a checklist of all message features that are 
concerned by Level 1, and match it with what has 

actually been teted, (b) or by making sure that the 
test associated with each Level 1 Requirement has been 

used 
at least once (<condition> part successful?), (c) or by doing a more 
through comparison between

what 
the test driver has been fed (input test cases) and what is observed on the 
wire. note

that 
would have  some "service-conformance" aspect.

 

 

Just 
following up on Michael schemas (ebXMLCompare.xsd, ebXMLSet.xsd, 
ebXMLTestSuite.xsd):

 

4- 
Michael: could you give a quick overview / use case of how you see instances of 
these schemas being used?

(I 
assume that instances of ebXMLTestSuite.xsd will be input to a "comparison 
engine"?)

 

5- I 
assume these schemas will then be used to compare actual output / expected 
output, right?

(that 
could then help on issue #3-c above.)

 

6- 
Issue: We'll need to define how to compare schema instances, e.g. in some cases, 
some distance is allowed

with 
the reference document (e.g. optional element, order, different 
values...)

 

7- 
COnformance reports - should it be instance of an extended Requirements schema 
with test results?

 

-Regards,

 

jacques

  
-----Original Message-----
From: Jacques Durand 
  [mailto:]
Sent: Monday, March 18, 2002 3:42 
  PM
To: ''
Subject: 
  [ebxml-iic-msg] Welcome

  
Hi: 

  
Welcome to the ebxml-iic-msg sublist. 

  
This is where you will read or post mails about MS 
  conformance testing design, 
and more 
  generally about the "Conformance Testing Task Force" activities and 
  material. 

  
Regards, 

  
Jacques Durand 
ebXML IIC chair