[ebxml-iic-conform] Coverage: last attempt

Title: RE: [ebxml-iic] MS conformance task force

Michael, Matt:

 

About 
"specification coverage" attribute: 

my 
last attempt at making sense of this attribute in the annotated 
spec:

(I 
still believe that may "adjust" expectations from MS TC reviewers, and clarify 
our "contract" 

with 
users about the results of our test suite 
...)

 

COVERAGE:

 

FULL: 
The test requirement(s) that address this specification item, are a good 
indicator of conformance

 to the specification item, i.e. if an MSH 
passes a test case that implements properly this test requirement(s), 

there 
is very strong indication that it will behave similarly in all 
situations identified by the spec item.

 

PARTIAL: The test requirement(s) that address this 
specification item, are only a partial indicator of 
conformance

 to the specification item, i.e. if an MSH 
passes a test case that implements properly this test requirement(s), 

this 
indicates that it will behave similarly in only a subset of all 
situations identified by the spec item.

Possible reason maybe:

- the 
pre-condition(s) of the test requirement(s) only identify on purpose a subset of 
these situations, 

that 
can be reasonably tested.

- 
the occurrence of situations that 
match the pre-condition of a Test Requirementit is under control of the MSH 

(e.g. 
implementation-dependent) and out of the control of the testbed. I.e. the test will be verified only 
in case 

the situation occurs during testing, 
which we cannot control. 

 

NONE: this 
specification item cannot be tested even partially, at least with the Test 
Framework on which this

Conformance test 
suite is to be implemented, and under the test conditions that are 
assumed.

  
 

  
 

  
Regards,

  
 

  
Jacques