Skip to content

Add Python regression tests; eliminate shell portability workarounds#1438

Open
ecrist wants to merge 1 commit intofix/bugs-and-efficiencyfrom
feature/python-regression-tests
Open

Add Python regression tests; eliminate shell portability workarounds#1438
ecrist wants to merge 1 commit intofix/bugs-and-efficiencyfrom
feature/python-regression-tests

Conversation

@ecrist
Copy link
Copy Markdown
Member

@ecrist ecrist commented Mar 14, 2026

Summary

Adds unit-tests-pr1436.py — a Python 3 rewrite of the PR #1436 regression suite that requires no external binaries other than easyrsa and openssl.

Updates op-test.sh to prefer the Python version and fall back to the shell version only when Python 3 is unavailable.


Motivation

The original shell test script (unit-tests-pr1436.sh) had to work around the absence of standard POSIX utilities on Windows, where the EasyRSA distribution ships outdated GNU-for-Windows binaries. PR #1437 rewrote the shell script to avoid those tools using shell built-ins.

Python removes the problem entirely:

Problem Shell workaround Python solution
cat not reliable while IFS= read -r loop path.read_text()
ls not reliable glob for loop path.glob() / path.iterdir()
sleep absent on Windows $SECONDS busy-wait or SKIP subprocess.run(timeout=N) — kills automatically
/tmp absent on Windows TMPDIR/TEMP/TMP/CWD chain tempfile.mkdtemp()
mkdir -p flag issues single-level mkdir + CWD fallback tempfile.mkdtemp()
rm -rf may be absent best-effort with advisory message shutil.rmtree(ignore_errors=True)
POSIX shebang ignored on Windows N/A (shell already handles it) _easyrsa_cmd() prepends sh on win32

Design

unit-tests-pr1436.py — stdlib only, Python ≥ 3.6:

_find_easyrsa()       resolves ERSA_BIN env → easyrsa3/easyrsa → PATH
_easyrsa_cmd()        prepends 'sh' on Windows; bare path on Unix/macOS
Runner.run()          subprocess.run() with timeout; as_posix() paths
Runner.t_pass/fail/skip  result accounting + formatted output

The seven tests map 1-to-1 with the shell version:

Test How Python improves it
FIX-1-passphrase-eof-exits subprocess.run(stdin=DEVNULL, timeout=10) raises TimeoutExpired if the process hangs — no background jobs, no signals
FIX-2a/b/c/d set_var validation path.write_text() writes vars files; subprocess.run() checks exit code
FIX-3 passphrase comparison Same subprocess call as before, simpler capture
FIX-4 mktemp naming Path.glob("temp.*") replaces the ls + for-loop in shell

op-test.shrun_local_regression_tests() updated:

  1. Probe for python3 then python, verify version starts with Python 3
  2. If found and .py file present: run Python version
  3. Otherwise: fall back to .sh version
  4. Map -v/-vv op-test verb flags to Python's -v

Compatibility

Runner Python available? Test file used Expected
GitHub Actions Ubuntu yes (python3) .py 7 PASS
GitHub Actions macOS yes (python3) .py 7 PASS
GitHub Actions Windows yes (python3 via setup-python) .py 7 PASS
Minimal embedded Linux no .sh fallback 6 PASS, 1 SKIP
mksh/Win32 without Python no .sh fallback 7 PASS (SECONDS mode)

Test plan

  • python3 unit-tests-pr1436.py -v — 7 PASS on macOS/Linux
  • python3 unit-tests-pr1436.py version — prints version string, exits 0
  • sh op-test.sh -v — confirm Python path is taken, 7 PASS in log
  • Remove python3 from PATH temporarily, run sh op-test.sh -v — confirm fallback to .sh is logged
  • On Windows CI: wop-test.batop-test.sh → Python tests pass

🤖 Generated with Claude Code

unit-tests-pr1436.py reimplements the same seven regression tests
using Python 3 stdlib only (no third-party packages).  Python's
native primitives eliminate every portability workaround that the
shell version required:

  subprocess.run(stdin=DEVNULL, timeout=10)
    FIX-1's EOF/infinite-loop test becomes a single call.
    TimeoutExpired is a typed exception — no background processes,
    no signal juggling, no SECONDS busy-wait, no platform ifdefs.
    subprocess.run() kills the child process automatically when the
    timeout fires, on every platform including Windows.

  tempfile.mkdtemp()
    Returns a writable temp directory on every OS using the platform's
    native temp mechanism (TMPDIR on Unix, TEMP/TMP on Windows).
    No mkdir, no /tmp hard-coding, no fallback chain needed.

  shutil.rmtree()
    Portable recursive directory removal — no rm.exe dependency.

  pathlib.Path / Path.glob() / Path.iterdir()
    Path manipulation and directory listing without ls, find, or
    any external tool.

  path.write_text() / path.read_text()
    File I/O without cat or redirected printf.

  _easyrsa_cmd() detects Windows (sys.platform == 'win32') and
    prepends 'sh' to the easyrsa invocation, because the Windows
    kernel does not process POSIX shebangs.

op-test.sh: run_local_regression_tests() now prefers the Python
  version and falls back to the shell version only if Python 3 is
  absent.  The Python executable is detected by probing 'python3'
  then 'python' and checking that the reported version starts with
  'Python 3'.

The shell version (unit-tests-pr1436.sh) is retained as the fallback
and remains the correct choice for environments without Python (e.g.
minimal embedded Linux, some CI images).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant