Skip to content

Latest commit

 

History

History
113 lines (81 loc) · 3.76 KB

testing.md

File metadata and controls

113 lines (81 loc) · 3.76 KB

Testing Strategy

This document outlines how io4dolfinx is tested, covering both local development testing and the Continuous Integration (CI) process on GitHub Actions.

Coverage reports:

You will find the updated coverage reports for the latest version on main tested against stable and nightly versions of dolfinx at the following links:

Local Testing

The library uses pytest for testing. To execute the tests locally, you first need to install the library and its dependencies.

Installation for Testing

Install the library with the optional test dependencies to ensure you have packages like pytest, coverage, and ipyparallel

python3 -m pip install ".[test]"

Running Tests

To execute all tests in the repository, run:

python3 -m pytest .

Generating Test Data

Some tests require specific datasets to verify compatibility with older software versions.

Testing against data from legacy dolfin

Some tests check the capability of reading data created with the legacy version of DOLFIN. To create this dataset, start a docker container with legacy DOLFIN, for instance:

docker run -ti -v $(pwd):/root/shared -w /root/shared --rm ghcr.io/scientificcomputing/fenics:2024-02-19

Then, inside this container, call:

python3 ./tests/create_legacy_data.py --output-dir=legacy

Testing against data from older versions of adios4dolfinx

Some tests check the capability to read data generated by adios4dolfinx<0.7.2. To generate data for these tests use the following commands:

docker run -ti -v $(pwd):/root/shared -w /root/shared --rm ghcr.io/fenics/dolfinx/dolfinx:v0.7.3

Then, inside the container, call:

python3 -m pip install adios4dolfinx==0.7.1
python3 ./tests/create_legacy_checkpoint.py --output-dir=legacy_checkpoint

Continuous Integration (GitHub Actions)

The repository relies on several GitHub Actions workflows to ensure code quality and compatibility across different environments.

1. Main Test Suite (test_package.yml)

This is the primary workflow triggered on pushes to main, pull requests, and scheduled nightly runs. It runs on ubuntu-24.04 using the official DOLFINx docker container .

Workflow Steps:

  1. Linting & Formatting: Checks code style using ruff and type consistency with mypy .
  2. Data Generation: * Creates legacy DOLFIN data using the create_legacy_data.yml workflow.
    • Creates legacy adios4dolfinx checkpoints using create_legacy_checkpoint.yml.
  3. Test Execution:
    • Installs the package with MPI-enabled h5py.
    • Runs the standard test suite with coverage.
    • Runs parallel tests using mpirun -n 4 ... mpi4py -m pytest.
  4. Reporting: Combines coverage reports and uploads them as artifacts.

2. Compatibility Testing

To ensure broad support, specific workflows test against different configurations:

  • MPI & ADIOS2 Versions (test_package_openmpi.yml):

    • Tests against both openmpi and mpich implementations using the ghcr.io/fenics/test-env containers.
    • Verifies compatibility with different ADIOS2 versions (e.g., v2.10.2, v2.11.0) .

  • Operating System (test_redhat.yml):

    • Runs the full test suite inside a RedHat-based container (docker.io/fenicsproject/test-env:current-redhat) to guarantee functionality on non-Debian systems .

3. Documentation (build_docs.yml)

Ensures the documentation builds correctly with jupyter-book on every push and pull request, preventing documentation regressions.