Scott,
The order in which we have developed
the conformance testkit has been has been to first write the test cases
for those assertions that can be tested by simply reviewing the log of
all messages having flowed between the Consumer and Producer. These are
just about all implemented at this point. The next task is to develop a
specialized portlet that can drive the interactions such that most of the
rest of the assertions can be tested. Once this portlet is available, we
will want to review the current plans for exploiting it to drive a sequence
of tests which define the test suite for a particular conformance level.
I doubt we will be able to get down
to a single button press to run a complete test suite as they all will
need to start with an invocation of getServiceDescription, but it should
get pretty close.
As a status update, Julie is currently
working on integrating/testing the newest test case code in preparation
for distributing a refresh of the testkit on alphaWorks. In parallel she
is pressing the lawyers for the approvals required to release the test
cases as an open source project. Our long term goal is that the testkit
be available for anyone to use to test the conformance level of an implementation.
Rich
"Goldstein, Scott"
<>
09/09/2004 05:18 PM
To
<>
cc
"Coco, Christopher"
<>
Subject
[wsrp-conformance] Long term
plans of Confomance Toolkit
I had some questions about the long term
plans of the toolkit. I downloaded it the other day to learn more
about it and give it a try. What I learned, was that there doesn’t
seem to be a way for me to run the toolkit such that I can make the claim
that I have a conformant implementation. In other words, because
it monitors interaction between the consumer/producer without providing
a strict set of actions to take in order to run all of the tests, it seems
very difficult to ensure that I have run all of them. Ideally, there
should be a user interface which contains a single button which, when pressed,
runs all tests in the kit and when it’s complete, states whether or not
the implementation passed all tests. For usability purposes, it would
be nice as well to be able to run tests cases on an individual basis.
Is such a mechanism in the long terms plans
for the conformance toolkit? Is the intention of the toolkit to provide
a way for a vendor to verify conformance or is it merely a tool that the
vendor can use during their QA process to catch possible interoperability
issues?
Scott
