One comment inlined below.
-Anish
--
Mike Edwards wrote:
>
> Anish,
>
> Reply inline...
>
> Yours, Mike.
>
> Strategist - Emerging Technologies, SCA & SDO.
> Co Chair OASIS SCA Assembly TC.
> IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
> Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431
> Email:
>
> Anish Karmarkar <> wrote on 03/03/2009 07:46:07:
> > Re: [sca-assembly] Issue 101: Complete the Conformance Section -
> > Comments on Proposal
> > Mike Edwards wrote:
> > >
> > > Folks,
> > >
> > > Some comments on the proposal:
> > >
> > >
> > > 1) I think that making a series of SCA-related documents into
> > > conformance points, as per section 12.1, is unnecessary and unwise.
> > >
> > > The only point of making these documents into conformance points is if
> > > it is intended to write testcase(s) that will validate those
> > > documents. I do not believe that we have the resources to write such
> > > testcases and as a result, the conformance demands made
> > > here are a waste of time and effort.
> > >
> >
> > I would like to understand what additional tests will have to be written.
> > There are already requirements on SCA runtime regarding rejecting what I
> > would call non-conformant SCA documents. To test that we would have to
> > create non-conformant documents, use it with a runtime and ensure that
> > the runtime rejects them. What additional test artifact would have to be
> > created wrt document conformance? I.e., we have a set of documents that
> > we assert are non-conformant. We of course would have a set of
> > conformant documents (for the positive test for a runtime).
> >
> > I can see us not wanting to create non-conformant (or conformant) SCA
> > documents for every little feature and element/attribute that schema
> > allows. But we are now talking about how detailed our test suite is
> > going to be. Given that there is a requirement for runtimes to reject
> > invalid SCA documents, this is already an issue. I don't think we need
> > to Turing complete.
> >
>
> The tests you describe above are simply tests of an SCA runtime - and we
> have
> these already, including "invalid documents". However, they only check on
> the valid behaviour of an SCA runtime.
>
> If we make claims about documents themselves being valid or invalid, it is
> my opinion that such claims are pretty worthless unless there are tests to
> back them up. Such tests would actually have to be able to accept some
> random document and evaluate whether that document was a valid document or
> not. I even heard one person on a previous call ask for a tool of exactly
> this kind.
But that isn't a test that is a document validator.
I don't see the need for a document validator defined by the TC.
It would be a nice tool to have around, but not required IMHO.
>
> At the moment I don't think it is worth the time and effort of building
> tests that could validate documents. As a result, I take the view that
> there is little to be gained in having documents as conformance points.
> The specification concentrates on what an SCA runtime does with a document
> and we have the tests to check out that the runtimes are doing the right
> things.
>
> Back to answer Martin's question of whether the runtime is right or whether
> the document is right when some failure occurs. If the SCA runtime passes
> our tests then the first reaction should be to say that the runtime is
> right and the document is incorrect. If this isn't the case, it actually
> shows a hole in our SCA runtime tests - go plug the hole!!
>
>
> > -Anish
> > --
> >
> <snip>
>
>
> ------------------------------------------------------------------------
>
> /
> /
>
> /Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/
>
>
>
>
>
>
