# DTS - Suite of Test Cases

.gitignore

*~

/dist
/output
__pycache__

about.md

Driver for making output for the DTS community test suite

Test cases are in cases.tsv:

• tab separated values
• one row per test case
• for multiple test cases with the same resource have multiple rows with the same resource
• at least three columns should be present on a row
  • resource
  • project
  • endpoint
• The subsequent columns are used for parameter values. They should make sense for the endpoint.

Usage

Show help:

python3 run.py -h

Run tests:

cat cases.tsv | python3 run.py

Or using -i param:

python3 run.py -i cases.tsv

Preparation

You need implement two shell scripts navigation.sh and document.sh or provide two commands with the --document or --navigation parameter. The scripts or commands are expected to return the endpoint's output on stdout and error messages on stderr.

You can choose between different types of parameter styles with the -p option. Use the --shell switch to call the commands via the system shell.

Example

Example for DTS Transformations:

document.sh

#!/bin/sh

SCRIPT=$(realpath "$0")
SCRIPTPATH=$(dirname "$SCRIPT")

$SCRIPTPATH/target/bin/xslt.sh -config:$SCRIPTPATH/test/saxon.xml -xsl:$SCRIPTPATH/xsl/document.xsl -it -ea:on $@

navigation.sh is the same, but uses an other XSLT stylesheet.

Running the test suite this like so:

python3 run.py -i cases.tsv -d '../document.sh' -n '../navigation.sh' -p equals

Alternative without shell scripts:

python3 run.py -i cases.tsv \
		-d '../target/bin/xslt.sh -config:../test/saxon.xml -xsl:../xsl/document.xsl -ea:on -it' \
		-n '../target/bin/xslt.sh -config:../test/saxon.xml -xsl:../xsl/navigation.xsl -ea:on -it' \
		-p equals \
		--shell

Help

Run python3 run.py -h to get help:

Output as of 2025-10-22:

usage: run.py [-h] [-i FILE] [--delim CHARACTER] [-n COMMAND] [-d COMMAND] [-o DIRECTORY] [-p FUNCTION_NAME] [-r FUNCTION_NAME] [--log FILE] 
 
Make output for the DTS community test cases running the configured implementation
 
options:
  -h, --help            show this help message and exit
  -i, --infile FILE     CSV/TSV file with test cases, one case per line. Defaults to stdin
  --delim CHARACTER     Delimiter. Default to TAB for TSV input.
  -n, --navigation COMMAND
                        Command to be called for a test case on the navigation endpoint. Defaults to ./navigation.sh
  -d, --document COMMAND
                        Command to be called for a test case on the document endpoint. Default to ./document.sh
  -o, --outputDir DIRECTORY
                        Base directory of the output files. Default to: output
  -p, --parameterStyle FUNCTION_NAME
                        How parameters are passed to the endpoint commands. Allowed values: ['equals', 'query', 'bash', 'java']. Default to: equals
  -r, --driver FUNCTION_NAME
                        How the endpoint command is called. Allowed values: ['cmd', 'shell']. Defaults to: cmd
  --log FILE            Log file. Defaults to stderr
 
For more info see

Output

There are two files per test case:

• .json for navigation endpoint or .xml for document endpoint holds the result
• .log has an error log

Output file tree as of 2025-10-21:

output/
├── dts-transformations
│   ├── document?resource=john.xml&start=p.1.start&end=p.1.end&tree=page-hateoas.log
│   ├── document?resource=john.xml&start=p.1.start&end=p.1.end&tree=page-hateoas.xml
│   ├── navigation?resource=john.xml.json
│   ├── navigation?resource=john.xml.log
│   ├── navigation?resource=john.xml&tree=page-content-by-intersection-2.json
│   ├── navigation?resource=john.xml&tree=page-content-by-intersection-2.log
│   ├── navigation?resource=john.xml&tree=page-content-by-intersection.json
│   ├── navigation?resource=john.xml&tree=page-content-by-intersection.log
│   ├── navigation?resource=john.xml&tree=page-content-xquery-like.json
│   ├── navigation?resource=john.xml&tree=page-content-xquery-like.log
│   ├── navigation?resource=john.xml&tree=page-hateoas.json
│   ├── navigation?resource=john.xml&tree=page-hateoas.log
│   ├── navigation?resource=john.xml&tree=page-level2-start-end.json
│   ├── navigation?resource=john.xml&tree=page-level2-start-end.log
│   ├── navigation?resource=john.xml&tree=page-milestone.json
│   ├── navigation?resource=john.xml&tree=page-milestone.log
│   ├── navigation?resource=john.xml&tree=wadm.json
│   └── navigation?resource=john.xml&tree=wadm.log
└── MyDapytains
    ├── navigation?resource=base_tei.xml.json
    ├── navigation?resource=base_tei.xml.log
    ├── navigation?resource=double_tree_lb.xml.json
    ├── navigation?resource=double_tree_lb.xml.log
    ├── navigation?resource=generated_lb.xml.json
    ├── navigation?resource=generated_lb.xml.log
    ├── navigation?resource=lb_diff_ab.xml.json
    ├── navigation?resource=lb_diff_ab.xml.log
    ├── navigation?resource=lb_same_ab.xml.json
    ├── navigation?resource=lb_same_ab.xml.log
    ├── navigation?resource=lb_uneven_ab_ending_node.xml.json
    ├── navigation?resource=lb_uneven_ab_ending_node.xml.log
    ├── navigation?resource=lb_uneven_ab.xml.json
    ├── navigation?resource=lb_uneven_ab.xml.log
    ├── navigation?resource=multiple_tree.xml.json
    ├── navigation?resource=multiple_tree.xml.log
    ├── navigation?resource=multiple_tree.xml&tree=alpha.json
    ├── navigation?resource=multiple_tree.xml&tree=alpha.log
    ├── navigation?resource=simple_doc.xml.json
    ├── navigation?resource=simple_doc.xml.log
    ├── navigation?resource=tei_with_two_traversing_with_n.xml.json
    ├── navigation?resource=tei_with_two_traversing_with_n.xml.log
    ├── navigation?resource=tei_with_two_traversing.xml.json
    ├── navigation?resource=tei_with_two_traversing.xml.log
    ├── navigation?resource=test_citeData_two_levels.xml.json
    ├── navigation?resource=test_citeData_two_levels.xml.log
    ├── navigation?resource=test_citeData.xml.json
    ├── navigation?resource=test_citeData.xml.log
    ├── navigation?resource=xml_entity_tail.xml.json
    ├── navigation?resource=xml_entity_tail.xml.log
    ├── navigation?resource=xml_entity.xml.json
    └── navigation?resource=xml_entity.xml.log

License

Choose one of MIT or C0

cases.tsv

driver.py

import subprocess


def cmd(command, outfname, errfname, kase):
    """Call command directly. The command name and the parameters are
    passed as a list of strings, like in ["ls", "-l" ].

    """
    with open(outfname, "w+") as outf, open(errfname, "w+") as errf:
        subprocess.run(command, stdout = outf, stderr = errf, shell = False, check = False)


def shell(command, outfname, errfname, kase):
    """Call command via the system shell. The command name and the
    parameters are passed as a single string, like in "ls -l".

    """
    with open(outfname, "w+") as outf, open(errfname, "w+") as errf:
        subprocess.run(" ".join(command), stdout = outf, stderr = errf, shell = True, check = False)



DRIVER_FUNCTIONS = [ cmd, shell ]

params.py

NO_PARAMS = ["endpoint", "project", "#project"]


def equals(kase, quote = ""):
    """Makes parameters of style key1=value1 key2=value2 ...
    """
    rc = []
    for k, v in kase.items():
        if k in NO_PARAMS or not v: continue
        if k.startswith("#"):
            k = k[1:]
        rc += [k + "=" + v + quote]
    return rc


def query(kase, quote = ""):
    """Makes query-style parameters ?key1=value1&Dkey2=value2 ...
    """
    rc = "?"
    i = 0
    for k, v in kase.items():
        if k in NO_PARAMS or not v: continue
        if k.startswith("#"):
            k = k[1:]
        if i != 0:
            rc += "&"
        rc += k + "=" + quote + v + quote
        i += 1
    return [rc]


def bash(kase, quote = ""):
    """Makes bash-style parameters: --key1 value1 --key2 value2 ...
    """
    rc = []
    for k, v in kase.items():
        if k in NO_PARAMS or not v: continue
        if k.startswith("#"):
            k = k[1:]
        rc += ["--" + k, quote + v + quote]
    return rc



def java(kase, quote = ""):
    """Makes parameters in Java-properties style -Dkey1=value1 -Dkey2=value2 ...
    """
    rc = []
    for k, v in kase.items():
        if k in NO_PARAMS or not v: continue
        if k.startswith("#"):
            k = k[1:]
        rc += ["-D" + k + "=" + quote + v + quote]
    return rc


PARAM_FUNCTIONS = [ equals, query, bash, java ]

run.py

import argparse
import csv
import sys
import os

# The functions do not need to be in scope! We only access them via the function lists.
from params import PARAM_FUNCTIONS, NO_PARAMS
from driver import DRIVER_FUNCTIONS

param_func_names = [ f.__name__ for f in PARAM_FUNCTIONS ]

param_func_dict = dict(zip(param_func_names, PARAM_FUNCTIONS))

driver_func_names = [ f.__name__ for f in DRIVER_FUNCTIONS ]

driver_func_dict = dict(zip(driver_func_names, DRIVER_FUNCTIONS))


PROJECT_URL = ""


cli = argparse.ArgumentParser(
    prog = os.path.basename(__file__),
    description = "Make output for the DTS community test cases running the configured implementation",
    epilog = "For more info see " + PROJECT_URL)

cli.add_argument(
    "-i",
    "--infile",
    metavar = "FILE",
    type = argparse.FileType("r"),
    default = sys.stdin,
    help = "CSV/TSV file with test cases, one case per line. Defaults to stdin")

cli.add_argument(
    "--delim",
    metavar = "CHARACTER",
    default = "\t",
    help = "Delimiter. Default to TAB for TSV input.")

cli.add_argument(
    "-n",
    "--navigation",
    metavar = "COMMAND",
    default = "./navigation.sh",
    help = "Command to be called for a test case on the navigation endpoint. Defaults to ./navigation.sh")

cli.add_argument(
    "-d",
    "--document",
    metavar = "COMMAND",
    default = "./document.sh",
    help = "Command to be called for a test case on the document endpoint. Default to ./document.sh")

cli.add_argument(
    "-o",
    "--outputDir",
    metavar = "DIRECTORY",
    default = "output",
    help = "Base directory of the output files. Default to: output")


cli.add_argument(
    "-p",
    "--parameterStyle",
    metavar = "FUNCTION_NAME",
    default = param_func_names[0],
    choices = param_func_names,
    help = "How parameters are passed to the endpoint commands. Allowed values: " + str(param_func_names) + ". Default to: " + param_func_names[0])

cli.add_argument(
    "-r",
    "--driver",
    metavar = "FUNCTION_NAME",
    default = driver_func_names[0],
    choices = driver_func_names,
    help = "How the endpoint command is called. Allowed values: " + str(driver_func_names) + ". Defaults to: " + driver_func_names[0])

cli.add_argument(
    "--log",
    metavar = "FILE",
    default = sys.stderr,
    type = argparse.FileType("w+"),
    help = "Log file. Defaults to stderr")


def run(input, navigation, document, paramfun = PARAM_FUNCTIONS[0], driverfun = DRIVER_FUNCTIONS[0], caseParamDelim = "\t", outdir = "out", logger = sys.stderr):
    cases = csv.DictReader(input, delimiter = caseParamDelim)
    for kase in cases:
        print(kase, file = logger)
        dir = os.path.join(outdir, kase["#project"])
        out_basename = os.path.join(dir, outfile(kase))
        os.makedirs(dir, exist_ok = True)
        if kase["endpoint"] == "navigation":
            proc = [navigation] + paramfun(kase)
            print(proc, file = logger)
            driverfun(proc, out_basename + ".json", out_basename + ".log", kase)
        elif kase["endpoint"] == "document":
            proc = [document] + paramfun(kase)
            print(proc, file = logger)
            driverfun(proc, out_basename + ".xml", out_basename + ".log", kase)
        else:
            print("unknown endpoint: " + kase["endpoint"])


def outfile(kase):
    """The name of the output file must not contain the full resource
    URI because of the slashes. We thus use the last segment of the
    path part as fake resource in the file name.

    ENDPOINT?resource=...&...

    The return value does not contain a suffix. It can be added by the
    calling function.

    """
    resource_basename = os.path.basename(kase["resource"])
    rc = kase["endpoint"] + "?resource=" + resource_basename
    for k, v in kase.items():
        if k in NO_PARAMS + ["#resource", "resource"] or not v: continue
        if k.startswith("#"):
            k = k[1::]
        rc += "&" + k + "=" + v
    return rc


if __name__ == "__main__":
    args = cli.parse_args()
    print(args, file = args.log)
    run(args.infile,
        args.navigation,
        args.document,
        paramfun = param_func_dict[args.parameterStyle],
        driverfun = driver_func_dict[args.driver],
        caseParamDelim = args.delim,
        outdir = args.outputDir,
        logger = args.log)