CTest/CDash without CMake¶

In general, four files are needed to test and submit to a CDash dashboard:

CTestConfig.cmake
CTestCustom.cmake
CTestTestfile.cmake

These files can be generated by hand. The detailed contents and format of the above files is beyond the scope of this documentation but can be found in Kitware's documentation.

NERSC provides a Python package PyCTest to streamline the generation of the above files. This package enables easily wrapping any build system into a CDash submission without CMake and generate tests. For example:

Makefiles
- pyctest.pyctest.CONFIGURE_COMMAND = "./configure"
- pyctest.pyctest.BUILD_COMMAND = "make -j8"
Python Project
- pyctest.pyctest.BUILD_COMMAND = "python setup.py install"

Using PyCTest¶

Name	Version	Platforms	Downloads

Install pyctest from Anaconda or PyPi

conda install -c conda-forge pyctest
python -m pip install -v pyctest

PyCTest does not simply implement routines for writing the necessary files, instead the package creates direct bindings to the CMake/CTest source code. The Anaconda distributions are pre-compiled and recommended. The PyPi distribution is a source distribution that will require building CMake from scratch -- which can take a significant amount of time depending on your system specifications.

Write a primary driver script for the project, e.g. pyctest-runner.py, and place in the top-level directory of the project.

PyCTest Example for Python Project¶

#!/usr/bin/env python

import os
import sys
import platform
import pyctest.pyctest as pyctest
import pyctest.helpers as helpers

parser = helpers.ArgumentParser("ProjectName", source_dir=os.getcwd(),binary_dir=os.getcwd(), vcs_type="git")
args = parser.parse_args()

pyctest.BUILD_NAME = "{}".format(args.build)
pyctest.BUILD_COMMAND = "python setup.py install"

test = pyctest.test()
test.SetName("unittest")
test.SetCommand(["nosetests"])

pyctest.run()

PyCTest Example for autotools Project¶

#!/usr/bin/env python

import os
import sys
import platform
import multiprocessing as mp
import pyctest.pyctest as pyctest
import pyctest.helpers as helpers

parser = helpers.ArgumentParser("ProjectName", source_dir=os.getcwd(), binary_dir=os.getcwd(),
                                vcs_type="git")
parser.add_argument("-n", "--build", type=str, required=True, help="Build name for identification")
args = parser.parse_args()

# CONFIGURE_COMMAND can only run one command so if autogen is required, just execute it here
cmd = pyctest.command(["./autogen.sh"])
cmd.SetWorkingDirectory(pyctest.SOURCE_DIRECTORY)
cmd.SetErrorQuiet(False)
cmd.Execute()

pyctest.BUILD_NAME = "{}".format(args.build)
pyctest.CONFIGURE_COMMAND = "./configure"
pyctest.BUILD_COMMAND = "make -j{}".format(mp.cpu_count())

# alternate test declaration format
pyctest.test("unittest", ["./run-testing.sh"])

pyctest.run()

PyCTest Example for CMake Project¶

#!/usr/bin/env python

import os
import sys
import platform
import multiprocessing as mp
import pyctest.pyctest as pyctest
import pyctest.helpers as helpers

project = "PyCTestDemo"
binary_dir = os.path.join(os.getcwd(), "{}-build".format(project))
parser = helpers.ArgumentParser("ProjectName", os.getcwd(), binary_dir)
parser.add_argument("-n", "--build", type=str, required=True, help="Build name for identification")
args = parser.parse_args()

pyctest.BUILD_NAME = "{}".format(args.build)
pyctest.UPDATE_COMMAND = "git"
pyctest.CONFIGURE_COMMAND = "cmake {}".format(pyctest.SOURCE_DIRECTORY)
pyctest.BUILD_COMMAND = "cmake --build {} --target all -- -j{}".format(
    pyctest.BINARY_DIRECTORY, mp.cpu_count())

pyctest.test("unittest", ["./run-testing.sh"])

pyctest.run()

PyCTest Example for CMake Project¶

Write a primary driver script for the project, e.g. pyctest-runner.py, and place in the top-level directory of the project.

#!/usr/bin/env python

import os
import sys
import platform
import multiprocessing as mp
import pyctest.pyctest as pyctest
import pyctest.helpers as helpers

project = "PyCTestDemo"
binary_dir = os.path.join(os.getcwd(), "{}-build".format(project))
parser = helpers.ArgumentParser("ProjectName", os.getcwd(), binary_dir)
parser.add_argument("-n", "--build", type=str, required=True, help="Build name for identification")
args = parser.parse_args()

pyctest.BUILD_NAME = "{}".format(args.build)
pyctest.UPDATE_COMMAND = "git"
pyctest.CONFIGURE_COMMAND = "cmake {}".format(pyctest.SOURCE_DIRECTORY)
pyctest.BUILD_COMMAND = "cmake --build {} --target all -- -j{}".format(
    pyctest.BINARY_DIRECTORY, mp.cpu_count())

pyctest.test("unittest", ["./run-testing.sh"])

pyctest.run()