Ideas for XSLT/Xpath Test Suite

Hi all:

 

    NIST is interested and in 
developing a test delivery system which may have 
an execution component to it.  This execution component is intended to be a tool to facilitate test execution and not 
an 

assestment of any processors.  NIST as well as 
the XSLT/Xpath committee

is not interested in assessing, evaluating, 
validating, certifying or otherwise 

measuring any implementation for the purposes 
of making any judgements.

 

Our past testing experience does indicate to us 
that "if you build they

will come" (from that famous movie).  It is 
our belief that the easier

it is for users to customize, view or execute 
the test suite, the more

likely will be that any test suite will be 
used.  

 

Other matters do require our attention at this 
point

------------------------------------------------------------------------------------------------------------------------

    Specifications file - I propose 
a file with the following format (pseudo -code)

 

        
    <TEST specifications = "XSLT or XPATH" category = "Test 

                       
category" set_switches = "YES or NO">

                 
<TEST_NAME>submitterX/.../testName.xsl</TEST_NAME>

                 
<TEST_INPUT>submitterX/.../inputFile.xml/</TEST_INPUT>

                 
<SPECS_SECTION>x.yy</SPECS_SECTION>

                 
<DESCRIPTION>Test description</TEST_DESCRIPTION>

                 
<EXPECTED_RESULTS>submittorX/.../results.html(or .out?)

     
            
</EXPECTED_RESULTS>

                 
</REMARKS>Any special remarks? or "N/A"</REMARKS>

            
</TEST>

 

        
where

             
submitterX  - name of  test submitter (i.e. NIST, Lotus, 
etc.)

             
specifications - XSLT or XPath accordingly

             
set_switches  - A "YES" or "NO" value that indicates whether or not this 
test

                       
            is prone to 
produce more than one behavior, i.e. this test may 

                       
            be removed 
during toggling.

 

             
TEST_NAME = a test name (including the file path) for the test 
file.

             
TEST_INPUT = path indicating where (or which) ".xml" file to use.

             SPECS_SECTION 
= section of specifications tested.

             
DESCRIPTION - textual description of test.

             
EXPECTED_RESULTS  = path of file containing vendor supplied 

                                                    expected  
results.

             
REMARKS - any special remarks pertaining to this test or "N/A" if 
none.

 

---------------------------------------------------------------------------------------------------------------

     Normative (human readable) 
HTML Test Report

 

                                                     
Test Report

 

                      Test 
Name: name

                      Specs. 
Section: section

                      
Description: test description

                      
Input Source - Clickable item to display input data on demand

                      
Expected Results - Clickable Item to display results on 
demand     

                                                  
 (expected output)

                      
Remarks: Any pertinent remarks to this test (present only if 
any

                                      
remarks are specified)

 

-------------------------------------------------------------------------------------------------------------

 

  Execution Test Results (Harness results, 
execution component)

 

                                                
Test Execution Results

                                                 
Processor: IUT

                                                 
Tested Subset - subset tested (see tests

                                                                            
categories below)

                                                   Date: 
The date

                      
Test Name: 
name
                      
Specs. Section: 
section
                      
Description: test 
description
                      
Input Source - Clickable item to display input data on demand

                      
Execution Results: string containing Processors 
results
                      
Expected Results - Clickable Item to display results on 
demand     

                                                   
(expected 
output)
                      
Remarks: Any pertinent remarks to this test (present only if 
any
                                      
remarks are specified

---------------------------------------------------------------------------------------------------------------------

 

    Normative XML rendition 
report

 

                        
should really look the same as the normative entire XML file

                        
(only smaller)

----------------------------------------------------------------------------------------------------------------

 

Test Categorization - As I suggested at the meeting 
last June 6, I think the following

system works for just about all possible tests 
(this pretty much follow the specs 

organization)

 

                   
                                    
XSLT Tests

 

                 
Stylesheet Structure Tests

                     
Namespace, Stylesheet element,  Literal result element,

                     
Qualified names, Forwards-compatible processing,

                     Combining 
stylesheets

 

                 Data 
Model Tests

                      
Root node children, Base URI,

                      
Unparsed  entities,  Whitespace stripping

                  

                  Template 
Tests

                      Template 
rules, Named templates

 

                   Result 
Tree Tests

                       
Creating elements and attributes,

                       
Creating text, Creating processing instructions,

                       
Creating comments, Copying, Computing generated text,

                       Numbering

 

                  
Data Manipulation Tests

                       
Repetition, Conditional processing

                       
Sorting, Variables and Parameters, Additional functions

     

                  
Extension Tests

                       
Extension elements, Extension functions

                  

                  Message-Fallback 
Tests

                       
Message, Fallback

 

                  Output 
Tests

                         
XML output, HTML output, Disabling output escaping

 

                     

                                            
               
Xpath Tests

 

                    Location 
Path Tests

                          Location 
steps, Axes, Nodes tests, 
Predicates,
                          Abbbreviated 
syntax

 

                   
Expression Tests

                          Basic, 
Function calls, Node-sets, Booleans,

                          
Numbers, strings, Lexical structure,

 

                   
Core Function Tests

                          Node 
set functions, String functions, Boolean functions,

                          
Number functions

 

                  
Data Model Tests

                          
Root node, Element nodes, Attribute nodes, Namespace

                          
nodes, PI nodes, Comment Nodes, Text Nodes

---------------------------------------------------------------------------------------------------------------------

 

Any comments, suggestions, changes etc to any of 
the above items are

welcome and open for discussion.  I know there 
are a number of other details

that must also be addressed, hopefully by July 6 
there will be none.

 

When I get back from Paris (week of June 19th), I 
will start developing some kind of a

working prototype (which may include parts of the 
harness) open for discussion

and hopefully get an agreement on these basic 
issues.

 

Greetings,

 

Carmelo 
Montanez