Section 7.1 of Test Framework revision

Title: call tomorrow Wed 8th, 3pm PT

Jacques,

 

  I sent this yesterday under a different 
"subject" heading (IIC Conf Call Tomorrow)..so some may have missed it.  
The bottom line with this modified section 7.1(see attached)  is that 
the

default behavior of <TestAssertion> has a 
dramatic effect upon how we must implement our test execution model.  If 
the default behavior of

the Test Driver (when it encounters a 
<TestAssertion>="false") result is to "exit/fail" the Test Case, this 
makes it very suitable for simple 

types of testing (ebMS, ebRegistry) where 
<Thread>s are not used and are not necessary.

 

I call this OPTION A:   

 

It may not be as suitable for BPSS 
testing.

 

OPTION B: (which you are suggesting)

 

 Changing the default behavior of the Test 
Driver to "abort" the current Thread when one of its <TestAssertion>s 
fails means that the subsequent <Join> operation reference that 
<Thread> 

 MUST also "abort" ITS parent 
<Thread> if it is not satisfied ( since we are in essence saying that a failed 
<TestAssertion> "failed/aborted" its container <Thread>.. and that 
the subsequent <Join> may not be possible

..depending upon whether it is an <AndJoin> 
or <OrJoin>). "Aborting" based upon the <Join> result makes 
sense, and would allow for the logic of OPTION A to "bubble up" through 
the

<Thread> hierarchy, ultimately achieving the 
same result.

  

For example, a <Thread> that listens for any 
received Error messages and finds one "aborts" if 
its <TestAssertion> (which verifies any received Error messages) 
fails. 

Therefore the subsequent <AndJoin> 
fails. Expanding upon that, the <AndJoin> could "abort" its container 
<Thread> .. and continue (aborting) up the execution tree until either 
no

<Join> is possible (exit/fail) or 
some possible continuation of execution flow is possible (based upon a 
satisfied  <OrJoin>).  

 

This is very similar to my initial proposal for a 
"cascading boolean chain" of  <Thread> 
results, where the boolean result of a lower <Thread> would 
"bubble" up the <Thread> hierarchy 

and execution would either continue or fail based 
upon logic flow. Here we are using <Join> 
as the determining operation for continued execution.

If we choose this approach, then section 7.1 MUST 
be re-written to show this change in logic.  My latest revision does not 
reflect this approach.

 

Additional rules would be:

 

If a <TestAssertion> fails in the "main" 
<Thread> ( which is <TestCase> ), then final <TestCase> result 
is "fail".. because the implicit <Join> of the main <Thread> is not 
possible.. since

that <Thread> is "aborted".

 

Failure of a <TestAssertion> in a serial 
<Thread> (i.e. a <Thread> declaration with an implied 
<Split><Join>) follows the same logic.  The "serial" 
<Thread> is aborted, and therefore its parent

<Thread> is also aborted ( since the implicit 
<Join> will fail).

 

This means that <Abort> MUST exist as an 
implicit (and therefore, I would guess "explicit" as well) operation for the 
Test Framework.  You had suggested removing it and only keeping 
<Return>.. which is the

symmetric opposite of <Abort>.  We can't 
do any of this without <Abort>.

 

My only concern with option B is based upon 
traceability of conformance/interoperability failure. We MUST be able to 
trace exactly

where the testing failure occured.  By 
eliminating "exit/fail" as the default behavior for a failed 
<TestAssertion> , we MUST be able to trace back to that failed 
<TestAssertion> which triggered the <Abort>.

  In the case of an <OrJoin>.. an 
<Abort> can actually happen in multiple <Thread>s. So there are 
multiple points of test failure.

 

Comments?

Mike

 

 

section7.1-JD1_MK_JD2_MK2.doc