Skip to content
35 changes: 35 additions & 0 deletions tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,41 @@ The corresponding `autest.sh` command is:

$ ./autest.sh --filter=something_descriptive

# Running tests in parallel

For faster test execution, a parallel test runner is available that distributes
tests across multiple workers. This is especially useful on machines with many
CPU cores.

$ python3 autest-parallel.py -j 16 --ats-bin <install>/bin --build-root <build-dir> --sandbox /tmp/autest-parallel

Key options:

* `-j N` - Number of parallel workers (default: number of CPU cores)
* `--ats-bin` - Path to the ATS install bin directory
* `--build-root` - Path to the build directory (for test plugins)
* `--sandbox` - Directory for test sandboxes (default: `/tmp/autest-parallel`)
* `-v` - Verbose output with real-time test progress per worker
* `--collect-timings` - Run tests individually to collect per-test timing data
* `--list` - List all tests and exit (useful for checking test discovery)

The parallel runner uses port offsets to ensure each worker gets a unique port
range, preventing conflicts between concurrent test instances. Tests known to
require serial execution (listed in `serial_tests.txt`) are run sequentially
after the parallel phase completes.

## Timing-based load balancing

If a `test-timings.json` file exists (generated by a previous run with
`--collect-timings`), the runner uses the Longest Processing Time (LPT)
algorithm to distribute tests across workers for balanced execution times.
Without timing data, tests are distributed round-robin.

## Adding serial tests

If a test cannot run in parallel (e.g., it uses hardcoded global resources),
add its path relative to `gold_tests/` to `serial_tests.txt`.
Copy link

Copilot AI Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new docs say to add a test’s path relative to gold_tests/ into serial_tests.txt, but the current implementation in autest-parallel.py strips entries down to a basename when matching. Either update the implementation to honor relative paths (preferred, avoids ambiguity), or adjust this documentation to reflect the actual matching behavior.

Suggested change
add its path relative to `gold_tests/` to `serial_tests.txt`.
add its test file name (basename, e.g., `something_descriptive.test.py`) to `serial_tests.txt` (matching is done by basename, not by path).

Copilot uses AI. Check for mistakes.

# Advanced setup

AuTest and the relevant tools can be install manually instead of using the wrapper script. By doing this, it is often easier to debug issues with the testing system, or the tests. There are two ways this can be done.
Expand Down
Loading
Loading