Fw: [members] OASIS Call for Participation: Test Assertions Guidelines (TAG) TC

Any ideas on whether this would be a
good fit for the spreadsheet test cases?  

-Rob

___________________________

Rob Weir

Software Architect

Workplace, Portal and Collaboration Software

IBM Software Group

email: 

phone: 1-978-399-7122

blog: http://www.robweir.com/blog/

----- Forwarded by Robert
Weir/Cambridge/IBM on 04/04/2007 08:45 AM -----

"Mary McRae"
<> 

04/03/2007 11:03 PM

Please respond to

<>

To

<>,
<>

cc

<>

Subject

[members] OASIS Call for Participation:
Test Assertions Guidelines (TAG) TC

To:  OASIS members & interested parties

   A new OASIS technical committee is being formed. The OASIS Test
Assertions Guidelines (TAG) Technical Committee has

been proposed by the members of OASIS listed below. The proposal, below,
meets the requirements of the OASIS TC Process

[a]. The TC name, statement of purpose, scope, list of deliverables, audience,
and language specified in the proposal

will constitute the TC's official charter. Submissions of technology for
consideration by the TC, and the beginning of

technical discussions, may occur no sooner than the TC's first meeting.

   This TC will operate under our 2005 IPR Policy [b]. The eligibility
requirements for becoming a participant in the TC

at the first meeting (see details below) are that:

   (a) you must be an employee of an OASIS member organization or
an individual member of OASIS;

   (b) the OASIS member must sign the OASIS membership agreement [c];

   (c) you must notify the TC chair of your intent to participate
at least 15 days prior to the first meeting, which

members may do by using the "Join this TC" button on the TC's
public page at [d]; and

   (d) you must attend the first meeting of the TC, at the time and
date fixed below.

Of course, participants also may join the TC at a later time. OASIS and
the TC welcomes all interested parties.

   Non-OASIS members who wish to participate may contact us about
joining OASIS [c]. In addition, the public may access

the information resources maintained for each TC: a mail list archive,
document repository and public comments facility,

which will be linked from the TC's public home page at [d].

   Please feel free to forward this announcement to any other appropriate
lists. OASIS is an open standards

organization; we encourage your feedback.

Regards,

Mary

---------------------------------------------------

Mary P McRae

Manager of TC Administration, OASIS

email:   

web: www.oasis-open.org 

[a] http://www.oasis-open.org/committees/process.php

[b] http://www.oasis-open.org/who/intellectualproperty.php  

[c] See http://www.oasis-open.org/join/ 

[d] http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tag

CALL FOR PARTICIPATION

OASIS Test Assertions Guidelines (TAG) Technical Committee

Name and abbreviation:

OASIS Test Assertions Guidelines (TAG) Technical Committee

Purpose:

The design of Test Assertions (TAs) associated with a specification

or standard - referred to in this charter as target specification -

has the following recognized benefits: (i) it improves the quality

of this specification during its design, and (ii) it reduces the

lead time necessary to create a test suite for this specification.

A test assertion (TA), also sometimes defined as test specification,

  is understood in this charter with the following general meaning:

A TA is an independent, complete, testable statement for

requirements  in the specification. A TA always refers to an item

under test (IUT), which is assumed to implement all or parts of the

target specification, so that this IUT is concerned with the

requirements addressed by the TA. This reference is either implicit

or explicit if it is necessary that the TA identifies the item under

test in some unambiguous manner. A TA describes the expected output

or behavior for the item under test within specific operation

conditions, in a way that can be measured or tested. A TA may refer

to a test harness architecture, of which a description limited to

the interactions between its components and the IUT may be

sufficient. Test assertions are generally different from test cases,

which are more detailed and contingent to a concrete test framework:

TAs are the basis to write test cases, and relate the latter to the

narrative of the target specification.

The general objective served by this TC is to facilitate the

creation  and usage of test assertions by any group involved in

designing a specification or standard of which software

implementations are expected to be developed, with a primary focus

on OASIS technical committees. The first step in achieving this is

to establish a common and reusable model, metadata, methodology and

representation for TAs.

This is aligned with the intent of the former OASIS Conformance TC,

although the focus in the current initiative is not on the various

aspects of quality and conformance, but on a specific one, namely

test assertions.

This TC also may submit its deliverables to the OASIS Technical

Advisory  Board (TAB) with the ultimate intent of having these

deliverables recommended to the OASIS Board of Directors for

contribution to a future revision of the OASIS TC Process,

rules aiming at improving the quality and adoption of OASIS

output. The TC would accept feedback and recommendations from the

TAB and OASIS Board.

The TC will also facilitate the promotion of its deliverables and

give them the visibility necessary to reach potential users in other

standard organizations.

Scope:

The scope of activity for this TC must be within the following topics:

  -- TA model: A model for designing Test Assertions (TA model),

that is independent from any particular target specification or

standard, but that may recognize different types of test

assertions, and may accommodate these in a specific way. Test

assertions may be for verifying conformance of an implementation to

a specification, or interoperability between implementations of the

same specification.  The TA model may address any useful

relationship identified between TAs, such as pre-requisites or

pre-conditions. It may include support for grouping several TAs - or

grouping entities - but will not pretend to fully model such

entities as conformance profiles, specification modules or

implementation roles.

  -- Test Environment modeling: Guidelines for characterizing the

test environment or test harness assumed by the test assertions, as

well as the item under test or IUT (an implementation of  all or

part of the target specification). Such characterization may remain

abstract by just focusing on the interaction between test

environment and IUT. This characterization may state the expected

properties and mode of operation required from the test environment

in order to verify the TAs. It may be seen as some of the

requirements for a real test harness intended to process test cases

based on these TAs.

  -- Related Notions: Within scope is the selection and/or

refinement of definitions of concepts expected to be related to TAs,

even if not  directly targeted by the modeling and methodology work

of this TC.  Such concepts may include: test case, conformance

profile or level, test environment / harness, test execution.

  -- Methodology: A methodology to make use of this model. This may

include examples derived from applying the above TA model to

particular specifications or standards.

  -- XML Markup: An XML representation for TAs and - if appropriate

- for their grouping entities. Additional notations supporting the

modeling of TAs (e.g. UML) are also within scope. The intent of the

  XML representation is left at the discretion of the TC. It could

be intended as an exchange format for editing tools, or as a source

for rendering/publishing, or as an input for a test tool, or a

combination of these.

  -- Case Studies: Investigation off current practices in other

OASIS TCs or other organizations which have already written test

assertions  for their specifications.

  -- Liaison and Promotion: Within scope are efforts to liaise with

other organizations than OASIS, and cooperation with external

contributors and groups that can provide input to the TC as well as

become users of its deliverables. Joint deliverables are within

scope.

Explicitly Out-of-scope:

  -- Defining test case metadata or representation. (Several

developments are occurring in this area.)

  -- Modeling or methodology of dimensions of variability such as

conformance profiles / levels / modules / discretionary items.

Substantial prior work exists in these areas. However the TC may use

this prior work as a foundation and may also accommodate these

constructs when designing an XML markup for TAs.

Deliverables:

The TC will produce the following deliverables, in this order:

(a) Test Assertions Guideline document, including definitions,

abstract model and structure not excluding the use of a modeling

notation (e.g. UML), methodology and examples showing how to extract

TAs from a target specification.

(b) A representation of TAs in XML, that refers to and supports the

TA model of (a).

Timeline:

The TC will aim at a Public Review draft of the Test Assertions

Guideline (a) before end of 2007. The guideline will be processed at

least to Committee Specification level, and up to OASIS standard if

considered appropriate by the TC.

The TC will aim at a Public Review draft of the TA XML Markup (b)

within first half of 2008. The TA Markup will be processed at least

to Committee Specification level, and up to OASIS standard if

considered appropriate by the TC

IPR Mode:

Royalty-Free on Limited Terms.

Audience:

The TC is welcoming any OASIS member with an interest and/or

experience in conformance, testing and quality assurance. Users of

the TC deliverables are expected to be members of any OASIS

technical committee.

Language:

English

=====

Complement to the Test Assertions Guideline Technical Committee

1. Related and Similar Work

The following standards and specification projects may be relevant

to the TC's deliberations. A common point in the following works,

is that they treat Test Assertions at best very lightly. Their

main focus is on other aspects of QA. In contrast, the TAG

objective is to exclusively emphasize the test Assertions, as

the single most critical factor of the testability of

specifications, and the  foundation to any conformance statement.

The TAG deliberately ignores other QA aspects to force users'

attention on TAs. The TC is interested in liaison with other

relevant open standards panels with a testing focus, but notes that

the efforts below mostly have concluded.

  (1) Conformance requirements for Specifications (OASIS, March

2002)

http://www.oasis-open.org/committees/download.php/305/conformance_requirements-v1.pdf

The focus is on Conformance statements, a small section is dedicated

to test Assertions. This is not sufficient for TAG charter, rather

complementary to it.

  (2) Conformance testing and Certification Framework (OASIS,

Conformance TC, June 2001)

http://www.oasis-open.org/committees/download.php/309/testing_and_certification_framework.pdf

is about testing guidelines and certification process. This concerns

steps that are subsequent to the TA definitions. In fact, proper TA

definitions are a prerequisite to this testing / certification phase.

  (3) QA Framework: Specification Guidelines (W3C, November 2004)

http://www.w3.org/TR/qaframe-spec/

This work is an exhaustive QA guideline, yet only makes a small

coverage of test Assertions, for which the reader does not get much

guidance. It does not convey the importance that TAs should have

according to this TAG charter. We see the TAG charter as a

concrete, detailed guidance to achieve a piece of what the QA

Framework recommends.

  (4) Variability in Specifications, WG note (W3C, 2005)

http://www.w3.org/TR/2005/NOTE-spec-variability-20050831/

Is a detailed guide on conformance technology, focused on a

technical aspect of conformance. A complement to [3], and an

in-depth study of some aspects defined in [1]. Same comments apply

as for [1].

  (5) Test Assertion Documents for WS-I profiles (2003-2005).

http://www.ws-i.org/Testing/Tools/2005/01/BP11_TAD_1-1.htm

Is an instance of test assertions for a particular standardization

work (here, WS-I profiles). While this work represents valuable

input, it is not a guideline for a larger number of specifications.

  (6) Test Metadata, QA Interest Group note, (W3C, September 2005)

http://www.w3.org/TR/2005/NOTE-test-metadata-20050914/

This work provides some ontology and terminology; most of it

appears to be relevant to TAs as well as to test development and

execution (the prime focus of the document). However it is no

substitute for modeling and methodology of TAs, but rather, a

terminology resource for a TA guideline.

2. First Meeting:

Time: Thursday May 3rd, 2:00 pm GMT/UTC

(3 pm UK, 4 pm Brussels/Wien, 10 pm Beijing, 11 pm Seoul/Tokyo, 7 am

San Francisco, 10 am Boston)

Duration:  90 minutes or less

Style: Conference call (dial-in info will be sent later)

Sponsor:  Fujitsu.

3. Projected Meeting Schedule:

Either 1h every week, or 2h every two weeks, depending on TC member

preferences. Assuming the first meeting is a conference call, a

face-to-face meeting will be set within 3 months of the first

meeting - June or July - once the work has actually started, to

maximize the value of face time.

4. Proposers:

   Hyunbo Cho, Korea Institute for Electronic Commerce,



   Patrick Curran, Sun, 

   Jacques Durand, Fujitsu, 

   Lynne Rosenthal, US National Institute of Standards and

Technology, 

   Victor Rudometov, Sun Microsystems, 

5. Convener:

Jacques Durand, Fujitsu

6. List of Contributions:

This work is anticipated to be contributed to this TC: Test

Assertion Guideline draft, (originally initiated within OASIS TAB,

2004-2005)

http://www.oasis-open.org/committees/document.php?document_id=20661&wg_abbrev=ebxml-iic

7. Proposed Working Title for deliverables:

Deliverable #1: Test Assertion Guideline - name: "TA Guideline".

Deliverable #2: Test Assertion XML representation - name: "TA Markup".

---------------------------------------------------------------------

This email list is used solely by OASIS for official consortium communications.

Opt-out requests may be sent to , however,
all members are strongly encouraged to maintain a subscription to this
list.