Comments on JMS test case document

Folks,

I've done a side-by-side comparison
of the JMS TA and TC documents, and have some comments. Unfortunately I
have not had time to review the test case artefacts against the TC document.

General comments about testing approach:

1) BJM_30001_TestCase - says that the
@uri must have valid syntax according to IETF spec....  how exhaustive
to we expect that to be (asked before)

2) There are a few normative statements
which include a prioritised list of cases.  Lets say the rule says
A then B then C.  The test cases are written to verify that behaviour
is as expected with A,  notA and B, notA and notB and C.  However
I don't believe there are tests for A and B and notC, A and notB and C,
A and B and C, notA and B and C.  As a specific example, we test that
@uri is used, and we test that if @uri is absent then <destination>
is used, but I'm not sure that we test that if both are present then @uri
is used.   Should we have tests that cover all combinations?

Specific test case comments:

BJM_30002_TestCase - do we need separate
cases for each of the resources that can appear in the @uri?

BJM_30010_TestCase - I believe the assertion
is untestable, but there is a corresponding testcase; not sure that it
really tests the statement

BJM_30019_TestCase - should there be
separate cases for destination element and @uri? two different failures

BJM_30024_1/2/3/4_TestCase - do we need
separate test cases for each header?

BJM_30024/5_TestCase - do we need separate
test cases for service response and reference request?

BJM_30031_TestCase - would have expected
to see a negative test here for a JCA 1.5 RA when the RA element is missing

BJM_40001_TestCase - what's it actually
testing?  That the response is as expected for the default wire formant?
 What about the different cases for the operation selector?

BJM_40002_TestCase - is this testing
the cases for the operation selector?

BJM_40004_TestCase - why negative...
shouldn't this test that the response is as expected?

BJM_40005T/B_TestCase - does this test
receiving requests at a service and responses at a reference?

BJM_40006_TestCase -  does this
test receiving requests at a service and responses at a reference?

BJM_40008_TestCase - I assume this is
the same as BJM_40002_TestCase, same comment.

BJM_40009_TestCase - same as BJM_40004_TestCase.

BJM_50002_TestCase - is there a negative
form of this when the assertion fails?

BJM_60003_TestCase - do we need separate
tests for the different correlationScheme values?

BJM_60006_TestCase - not sure how we
test each of the cases.

BJM_60007_TestCase - does this need
to include cases where there is or isn't a response/destination?

BJM_60009_TestCase - untestable, not
marked as such in the TA document

BJM_60010_TestCase - separate tests
needed for different correlationSchemes?  This really seems to duplicate
BJM30003/4/5/6.

BJM_60013_TestCase - separate tests
needed for 60004/60005 cases?

BJM_60014/14A - renamed for consistency
to BJM-TA-60014-1 and -2

BJM_60014_TestCase - separate tests
needed for destination identified by @uri or <destination>?

BJM_60017_TestCase - separate test cases
required for one-way and request/response operations? etc...

BJM-TA-60018 currently has no testcase
but not stated as being untestable.

Some comments about test cases that
I would want to verify against the test case artefacts.

BJM_30019_TestCase/BJM30022_TestCase
- need to see how the activation spec is checked for referring to the same
destination;

BJM_40011_TestCase - how is the invalid
resolved operation name achieved?  Are there variants on this?

Regards, Simon

Simon Holdsworth

STSM, SCA Bindings Architect; Master Inventor; OASIS SCA Bindings TC Chair,
AT&T and Boeing Lab Advocate

MP 211, IBM UK Labs, Hursley Park, Winchester SO21 2JN, UK

Tel +44-1962-815059 (Internal 245059) Fax +44-1962-816898

Internet - 

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598. 

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU