Test suite performance analysis (Linux container, with Postgres)

[jnasbyupgrade]

Summary

Full suite timing on Linux (podman container, PostgreSQL 17 + pg_tle available): ~37 seconds.

How to read the numbers

Times were collected by piping make test through ~/bin/timestamp, which prints the elapsed time until the next line appears alongside each output line. So the interval shown with ok N is the time test N+1 took to complete.

Major time consumers (>0.5s)

Cost	What
~3.8s	Foundation creation — done twice (once for recursion check, once for main run)
3.3s	pgtle: requires field becomes ARRAY when present — see below
~1.25s	pgtle: make clean removes pg_tle directory (make clean via PGXS)
~1.22s	pgtle: control file change triggers rebuild (sleep 1 + two make pgtle calls)
~1.22s	pgtle: SQL file change triggers rebuild (sleep 1 + two make pgtle calls)
~1.09s	Foundation test 7: git push to fake remote
~4s total	8 independent test env creations (~0.5s each: control-errors, dist-clean, doc, gitattributes, make-results, make-test, multi-extension, pgtle-install, update-setup-files)

The big one: pgtle: requires field becomes ARRAY when present (3.3s)

This test in test/sequential/04-pgtle.bats contains:

sleep 2
touch pgxntool-test.control
sleep 1

These 3 seconds of sleep are a deliberate guard against filesystems with coarse timestamp granularity (FAT32 = 2s, HFS+ = 1s). The pattern makes the control file's mtime definitively newer than the output file's, so make detects the change.

Tests 48 and 49 (control file / SQL file change triggers rebuild) solve the same problem with only sleep 1 each — they already hold the mtime from before the touch and compare after, so they only need 1 second. Test 44 could adopt the same pattern, saving 2 seconds.

However, there is new timestamp infrastructure coming in another branch that handles this more elegantly. This note is left here for that work to reference.

Context

This analysis was run after:

• PostgreSQL 17 started and pg_tle installed (previous runs had most postgres-dependent tests skipping/trivially passing in ~22s)
• With postgres running, the honest cost is ~37s for a full clean run

Remaining failures

5 tests in make-results.bats fail due to a separate issue: the test environment has no SQL files in test/sql/, so make test runs with an empty REGRESS list and produces no result files for make results to copy. This is being investigated separately (source file handling).