Re: [ebxml-iic] comments on use case #2, and on termination of test case

Title: comments on use case #2, and on termination of test case

 

  
----- Original Message ----- 

  
From: 
  Jacques 
  Durand 

  
To: 'Michael Kass' ; Jacques 
  Durand ; ebXML IIC - main list (E-mail) 
  (E-mail) 

  
Cc:  ;  
  

  
Sent: Tuesday, August 24, 2004 3:06 
  PM

  
Subject: RE: [ebxml-iic] comments on use 
  case #2, and on termination of test case

  

  
Mike:

  
 

  
That would 
  be satisfying: as long as we have a way to exit from a thread (abort) 
  without

  
exiting 
  from the test case.

  
The only 
  argument I'd have against your position, is that it is not always on the 
  "intuiton" side

  
(e.g. you 
  spawn an error-detection thread that will explicitly exit on failure as soon 
  as an error message is received  (<WhenTrue><Exit>Fail...) , 
  but if you forget the <WhenFalse>Continue,

  
- which 
  additionally may sound odd if at the end of a thread - then this thread wil 
  still Fail the entire test case...)

  
 

  
[MIKE3] - Understood.. this wasn't the best example I could 
  think of.. 

  
 

  
So we have 
  now 5 possible actions to follow a Test Assertion result (either True or 
  False):

  
- continue 
  (or Continue_Thread?) (default for Assertion=True)

  
- abort (or 
  Abort_Thread?)

  
- exit Fail 
  (default for Assertion=False)

  
- exit 
  Pass

  
- exit 
  Undetermined

  
 

  
Am I 
  right?

  
 

  
[MIKE3] - 
  Yes.  We are implmenting now.

  
 

  
Additionally:

  
- if the 
  test case terminates its execution (all threads?, or just main thread?) 
  without exiting either on Fail or Undetermined, then it 
  Passes.

  
[MIKE3] - 
  Add "exception" in there as well and yes

  
 

  
- If a 
  thread aborts, its container thread (or caller thread) 
  aborts.

  
[MIKE3] - 
  No.. why do we need to do this???  Why kill the parent/caller Thread in 
  which you may wish to do your join????  Also, why not let 
  logic

  
contiue to 
  a final resolution?

  
 

  
- if main 
  thread aborts: test case Fails? (abort is a logical meaning) or just 
  Undetermined?

  
[MIKE3]-We'll go with undetermined, and this will only occur at 
  the <TestCase> level ( treating TestCase as the "main" Thread).  It 
  simplifies scripting greatly,

  
achieves 
  the same goal.  Having a "main" Thread is only meaningful from a 
  semantic view..  TestCase is in f act the main 
  Thread.

  
Comments?  ( We need to move quickly on resolution of 
  this)

  
 

  
 

  
Jacques

  
 

  
 

  
 

  
    
-----Original Message-----
From: Michael Kass 
    [mailto:]
Sent: Monday, August 23, 2004 8:54 
    PM
To: Jacques Durand; ebXML IIC - main list (E-mail) 
    (E-mail)
Cc: ; 
    
Subject: Re: [ebxml-iic] comments on 
    use case #2, and on termination of test case

    
Jacques,

    
 

    
 

    
   Below are some of my thoughts 
    regarding "abort".    I think that it is a useful and 
    necessary function for concurrent Threads

    
that will later be "orJoined".

    
 

    
I think however, that <Abort> should be 
    explicitly set by the Test writer, since, in  99% of 
    <TestAssertion> operations in a typical Test Suite,

    
a boolean result of "false" for a 
    <TestAssertion> will signal a defaul exit  condition for the Test 
    Case, with a final result of "fail".  This is the case for

    
ebMS testing, and will likely be the case for 
    Registry testing as well. BPSS testing will of course have more complex, 
    concurrent Thread opterations

    
that would benefit this 
    functionality.  However, making <Abort> the default logical 
    branch on a failed <TestAssertion> does not represent the typical use 
    case

    
in our test scripting.

    
 

    
  Also, requiring the test writer to 
    "micro manage" every <TestAssertion> in the test suite, and explicitly 
    set a 
    <WhenFalse><Exit>fail</Exit></WhenFalse>

    
and 
    <WhenTrue><Continue/></WhenTrue>

    
is (in my opinion) against the intuitive 
    meaning of a <TestAsssertion> operation, where "true" = "pass" and 
    "false" = "fail".   Such

    
default behavior could be "overriden" in 
    the case of a (rare) "abort" situation.  Forcing explicit 
    declarations of branching will require unnecessary 

    
and labor-intensive micro-scripting for each 
    Test Case, where more intuitive default behavior rules for <TestAssertion> could handle the 
    majority

    
of test cases.

    
 

    
  I would like to propose 
    keeping the existing implied <TestAssertion> logic, but adding an 
    explicit <Abort> option for <TestAssertion> that would let the 
    test writer abort a

    
(concurrent) Thread if a particular <TestAssertion> fails.

    
 

    
 

    
Comments?

    
 

    
Mike

    
 

    
 

    
----- Original Message ----- 

    
      
From: 
      Jacques Durand 

      
To: ebXML IIC - main list 
      (E-mail) (E-mail) 

      
Cc: '' ; '' 
      

      
Sent: Monday, August 23, 2004 4:01 
      PM

      
Subject: [ebxml-iic] comments on use 
      case #2, and on termination of test case

      

      
Mike: 

      
I think we need to tighten the semantics of a Test 
      Assertion failing without explicit exit statement: 
I am concerned that these implicit rules we have will be 
      confusing on complex test cases that have concurrent threads...

      
(see my comment in 1.1.4, attached) 
I would suggest we consider the following proposal: 
      
- no "fail" outcome is produced unless an explicit 
      fail exit statement is met during exec. 
- no 
      "pass" outcome unless an explicit pass exit statement is met during 
      exec. 
- a failed assertion without explicit 
      exit statement, by default will "abort" the thread, but just the 
      thread. 
- a passed assertion without explicit 
      exit statement, by default will "continue" the thread. 
- when threads are joined, an aborted thread will automatically 
      cause failure of an and-join (which aborts the container thread). In case 
      it is an or-join, the aborted thread will just be ignored by the or-join 
      (the or-join will fail if all joined threads abort). If a thread that was 
      split but never joined, aborts, then it just stops and is simply ignored 
      for the rest of the test case exec and outcome. 

      
- if the Main thread of a test case aborts, the 
      outcome is "Undetermined" by default (this is the only case of implicit 
      outcome, in addition to other explicit "Undetermined" outcomes)

      
Some additional Comments starting p.5 of the attached 
      doc (mostly, use case #2). 

      
Also, I was searching for the section where we specify 
      the test step timeout (MaxDuration?) , in the draft spec, 
and did not find it (same for the "sleep" statement). 
      

      
Cheers, 

      
jacques 
<<section7.1-JD1_MK_JD2.doc>> 

      

      

      

To unsubscribe from this mailing list (and be removed from the 
      roster of the OASIS TC), go to 
      http://www.oasis-open.org/apps/org/workgroup/ebxml-iic/members/leave_workgroup.php.