Fw: TAB Comments on Test Assertions for SCA_J Common Annotations and APIsv1.1

Folks,

I am copying Jacques' reply to my email
to the SCA-J mailing list as Jacques is unable to post there directly.

Yours, Mike

Dr Mike Edwards
 Mail Point 146, Hursley
Park

STSM
 Winchester, Hants SO21
2JN

SCA & Services
Standards
 United Kingdom

Co-Chair OASIS SCA
Assembly TC
 

IBM Software Group
  

Phone:
+44-1962 818014
  

Mobile:
+44-7802-467431 (274097)
  

e-mail:

  

 

 

----- Forwarded by Mike
Edwards/UK/IBM on 16/02/2011 09:10 -----

From:
Jacques Durand <>

To:
Mike Edwards/UK/IBM@IBMGB

Cc:
""
<>, OASIS Java <>

Date:
15/02/2011 22:36

Subject:
RE: TAB Comments on Test Assertions
for SCA_J Common Annotations and APIs v1.1

Mike:

 

Thanks for your response.
Fair enough.

Indeed there are assumptions
your TC made (e.g. about the default “target” of the tests) , that help
understand better some of your TAs, once known.

My comments were more from
an outsider perspective – i.e. to make sure your TA doc is well understood
by external folks with little background. Nothing critical.

I think it is great that
you found the TAG work useful to produce such a significant corpus of test
assertions !

 

(please forward as appropriate
– I may not have posting rights)

 

Cheers,

Jacques

 

From: Mike Edwards [mailto:]

Sent: Monday, February 14, 2011 8:35 AM

To: Jacques Durand

Cc: ; OASIS Java

Subject: Re: TAB Comments on Test Assertions for SCA_J Common Annotations
and APIs v1.1

 

Jacques, 

The SCA J technical committee thanks you and the TAB TC for the comments
on the SCA_J Common Annotations and APIs V1.1 Test Assertions.

The following are responses to your comments:

Issue JAVA-217: [Comment 1] : weak design: Target is often not well identified,
in several TAs. 

JCA-TA-2005, JCA-TA-2006,
JCA-TA-2007 and some others.

It is stated that the target of these statements is incorrect and that
the "real" target should be an SCA runtime. 

First, an observation in general that the test assertions for pretty well
all of the SCA specifications are all about verifying 

the behaviour of some SCA runtime, so that we take the view that attempting
to define the runtime as the target is pretty 

well useless in defining the test assertions.  Instead, the SCA TCs
took the view that what the assertions need to do is to 

concentrate on "observables" - things that could be measured
in one way or another. 

So taking JCA-TA-2005 as an example of a wider class of assertions, here
the "observable" is the behaviour of a Java class 

that is marked in a particular way - some specific behaviour is expected
when the SCA runtime deals with the class, and it 

is this behaviour that is observable - the creation and initialization
in this case.  We believe that it is correct and useful to 

describe the assertion in terms of this observable behaviour (and indeed
we successfully created a testcase that probes 

for this behaviour). 

As a result of these considerations, the TC is not minded to make changes
to the assertions along the lines that you suggest. 

Similarly, shifting some of the material into the prerequisite statement
is not particularly helpful as suggested does not in these 

cases appear to improve the assertions - reporting on artifacts that don't
meet the prerequisites is not the aim of the testcases 

at all - the testcases are deliberately set up to have artifacts that meet
certain conditions to probe that such artifacts are then 

handled correctly when presented to the SCA runtime. 

Issue JAVA 218: [Comment 2] : (minor)
Target is often too fine-grain, leading to convoluted references.  

It is not clear to us that the references identified in JCA-TA-3005 are
in any way convoluted - the target is clearly identified as the

specific attribute of an element and the predicate is clearly a statement
about the value that the attribute must have. 

It is not clear to the TC that the offered alternative formulation is any
clearer than the original. 

Issue JAVA-219: [Comment 3] : some spec requirements may need more than
one TA 

In the case of the SCA specifications, there is a general blanket requirement
that if there are artifacts in error, then the runtime must raise

an error.  We have taken the view that in general test assertions
are not created for each normative statement merely to state that an

error must be raised by the runtime if the statement is violated.  In
many cases, negative testcases are written that do depend on the

runtime raising an error, but this is ancillary to the "positive"
form of the test assertion that is used.

As a result, the TC is not minded to create the second negative form of
the assertion that is proposed.

Issue JAVA-220:  [Comment 4] : Target should not be an "event",
but an identifiable entity. 

As for comment 1, it is taken as read that it is an SCA runtime is involved,
but again, the issue is what is observable.

In this case the client invocation is what is observable (what happens
in the runtime itself is unobservable, other than

its impact on the invocation).

The invocation may sound like an event, but in reality it is a completed
process - and so a measurable "thing" that either

conforms or not. 

As for splitting the assertion into separate parts for the parameters and
the returned value, that is something that a testcase

might choose to do, but it is not a necessary change for the test assertion.
 In reality I think our testcase checks for both at

the same time. 

Issue JAVA-221: [Comment 5] : stronger
use of "tags" might help classify TAs based on targets:  

It is possible that the tags could be improved, but the suggested ones
such as "SCA_Runtime_component" and "SCA_Runtime_client"

are not appropriate - these do not reflect anything in the specification.

In general, where a specific element such as <interface.java/> is
the target, then the tags do typically contain the name of the element.
- such 

as JCA-TA-3002 

Yours, Mike 

Dr Mike Edwards

 Mail Point 146, Hursley
Park 

STSM

 Winchester, Hants SO21
2JN 

SCA & Services
Standards 
 United Kingdom

Co-Chair OASIS SCA
Assembly TC 
 

IBM Software Group

  

Phone:

+44-1962 818014

  

Mobile:

+44-7802-467431 (274097)

  

e-mail:



  

 

 

 

 

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598. 

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU 

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598. 

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU