[ebxml-iic-conform] Test Requirement Coverage document, first iteration

Jacques and all,

    Here is the first version of the Test Requirements Coverage 
document.  It is annotated using the current
test requirement ID's, and the current "coverage" labels.
    Two key points before we submit it to the Messaging TC...

1) We will need to lay down some solid criteria for
defining coverage as "full", "partial" and "none".  The major issue ( in my 
opinion ) will have to do with how much our
test harness will be able to peer into and manipulate the internals of a 
candidate MSH.  I think that we need
a list of quantified criteria in order to make these coverage labels more 
meaningful.
2) This scheme seems to work well to identify exactly what has ( and has 
not ) been covered in the spec.  I see
more potential parts of the spec to cover, based upon the annotation. Due 
to limited time, it may be best to submit
as is and get feedback from the TC, rather than iterate more on the 
requirements.  Comments?

   Also attached are the updated requirements ( fixed typos, naming, a few 
duplicates ).  These have been checked into CVS.
Again, "coverage" column will be removed when we've "finalized" the 
requirements and pass them on to Matt for merging and
resequencing the numbers.  Right now, it is helpful to see  the coverage 
since we need to finalize that criteria.

   My idea for criteria for coverage is pretty simple:

1) "Full" = our test case leaves no doubt that this item in the spec has 
been tested fully ( this is where the vast majority of our test 
requirements fall )
2) "Partial" = Due to limitations ( either in the test service, the test 
party software or rigor in writing a test case for all potential 
possibilities ), this requirement could not be completely tested
3) "None" = Due to limitations in the test service, the test party software 
we could not test this requirement at all

   The main problem that I see is defining just what the capabilities of 
the test service and the test party software will be.  For example, will we 
be performing "interrupts" of
MSH service to test reliable messaging, will we be able to check digital 
signatures, will we be able to check message persistence on the candidate 
side?  These questions,
and more need to be answered before a reliable estimation of coverage can 
be made, in my opinion.

   Comments?

Mike