Skip to content

Modernize project metadata, swap to ruff, harden CI#61

Open
twoerner wants to merge 20 commits into
mainfrom
maint/modernize
Open

Modernize project metadata, swap to ruff, harden CI#61
twoerner wants to merge 20 commits into
mainfrom
maint/modernize

Conversation

@twoerner
Copy link
Copy Markdown
Collaborator

This branch modernizes the project metadata, swaps the formatter
from black to ruff, and hardens the CI workflow. It is split
into 20 small commits along clean logical lines so the reviewer can
walk the diff one concept at a time.

Project metadata

  • Fix the silent required-python -> requires-python typo in
    pyproject.toml; PEP 621 ignored the misspelled key, so the
    package floor was effectively unenforced before this change.
  • Drop Python 3.8 (end-of-life since October 2024) from the
    supported set; add Python 3.13 (October 2024) and 3.14 (October
    2025). The CI matrix moves with the same change so every declared
    interpreter is actually exercised.
  • Declare the license via PEP 639 (license = "GPL-2.0-only",
    license-files = ["LICENSE"]).
  • Add a maintainers = [...] block alongside the existing
    authors = [...] historical credit.
  • Add the Changelog URL to [project.urls].
  • Move the package version source into src/bmaptool/__init__.py
    (__version__); CLI.VERSION re-exports it so existing imports
    keep working. make_a_release.sh is updated to bump the new
    location.

Tests and dev extras

  • Remove tests/oldcodebase/ (~6500 lines of pre-Python-3
    BmapCopy modules) and the backward-compat half of
    tests/test_compat.py that consumed them. The forward-compat
    half (current BmapCopy exercised against every historical bmap
    fixture in tests/test-data/) is preserved.
  • Drop six from [project.optional-dependencies].dev since the
    backward-compat test is gone.
  • In tests/test_api_base.py, pin pbzip2 -p1 and pigz -p 1 so
    the fixture-generation step uses a single worker. The default
    worker-per-CPU behavior has been seen to OOM on shared CI
    runners; single-worker mode trades wall-clock time for memory
    while still exercising both producer paths and producing valid
    bzip2/gzip streams for the TransRead.py consumer paths to
    decompress.

Formatter swap

  • Replace black with ruff in [project.optional-dependencies].dev
    and add a [tool.ruff] block (line-length = 88,
    target-version = "py39").
  • Apply a one-time tree-wide ruff format reformat in its own
    commit, isolated so it can be added to .git-blame-ignore-revs
    and git blame skips past it.
  • Add .git-blame-ignore-revs at the repo root listing that
    reformat commit. GitHub honors the file automatically;
    contributors opt in locally with
    git config blame.ignoreRevsFile .git-blame-ignore-revs.

The ruff swap is scoped to formatting only on this branch. Enabling
[tool.ruff.lint] against the existing tree surfaces a substantial
backlog (~285 findings across pyupgrade, bugbear, and bandit rules)
that deserves a separate, focused triage rather than a rider on the
formatter swap.

CI hardening

  • Replace the psf/black@stable step with a ruff format --check .
    step pinned to a known ruff version, so the formatter check uses
    exactly the configuration in pyproject.toml.
  • Bump actions/checkout in the lint job from v3 (end-of-life,
    prints deprecation warnings) to v4 in its own commit.
  • Pin every third-party action (actions/checkout,
    actions/setup-python) by 40-character SHA with a trailing
    # vX.Y.Z comment so the resolved commit is part of this repo's
    controlled state instead of re-resolving on every workflow run.
  • Declare permissions: contents: read at workflow scope and on
    each job, so the GITHUB_TOKEN is no longer the default
    read-and-write scope.
  • Set strategy.fail-fast: false on the test matrix so a failure
    on one Python version does not cancel the rest; knowing whether
    a failure reproduces across all interpreters or only one is the
    first useful triage signal.
  • Add a non-blocking pip-audit step in the lint job that reads
    pyproject.toml and queries the OSV vulnerability database.
    continue-on-error: true keeps a new transitive-dep CVE from
    turning the workflow red; the findings are visible in the run
    log and can be acted on at the maintainer's cadence.
  • Add .github/dependabot.yml opening grouped weekly pull
    requests for github-actions and pip so SHA pins and
    dev-extras versions stay current without manual chasing.

Changelog

CHANGELOG.md carries a single Added / Changed / Removed entry
for the branch.

Verification

Locally (on the developer machine):

  • The full unittest suite passes (python3 -m unittest) with the
    new pbzip2 -p1 and pigz -p 1 flags in place; both producer
    paths still feed their TransRead.py consumer counterparts.
  • python3 -m build produces an sdist and a wheel whose METADATA
    reports Requires-Python: >=3.9, confirming the
    requires-python typo cannot regress without the smoke check
    catching it.
  • ruff format --check . is clean tree-wide.
  • actionlint and yamllint are clean on
    .github/workflows/ci.yml and .github/dependabot.yml.

The branch is split into small, self-contained commits; each
commit on its own leaves the tree in a working state.

twoerner added 20 commits May 21, 2026 20:44
The PEP-621 key is `requires-python` (with the `s`), not
`required-python`. Hatch and setuptools silently ignore unknown
top-level project keys, so the misspelled line has had no effect
since it was written: nothing in the build pipeline has actually
been enforcing a minimum Python version.

After this change, `python -m build` produces a wheel whose
METADATA includes `Requires-Python: >=3.8`; before it produced no
such line at all.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Python 3.8 reached end-of-life in October 2024; no upstream security
or bug fixes are issued for it any longer. Drop it from the supported
set.

Python 3.13 (released October 2024) and 3.14 (released October 2025)
are now both stable and broadly available; add them to the supported
set.

The change spans two surfaces that have to move together:

  - pyproject.toml: requires-python bumps to ">= 3.9" and the trove
    classifier list loses 3.8 and gains 3.13 and 3.14.
  - .github/workflows/ci.yml: the test matrix loses 3.8 and gains
    3.13 and 3.14, so every interpreter declared as supported is
    actually exercised by CI.

The bmaptool source code uses no 3.9+-specific syntax, so the floor
change is a packaging and CI declaration only; no source files need
to change to honor it.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
PEP-639 (accepted in 2023, supported by hatchling >= 1.18) introduces
a structured way to declare a project's license in `pyproject.toml`:
an SPDX expression string in the `license` field, plus a
`license-files` glob list pointing at the license text in the
distribution.

Adopt that form:

  license = "GPL-2.0-only"
  license-files = ["LICENSE"]

The existing trove classifier
"License :: OSI Approved :: GNU General Public License v2 (GPLv2)"
stays in place. PEP-639 marks license classifiers as deprecated but
encourages keeping them during the deprecation overlap so older
tooling that has not yet learned to read the SPDX field still
identifies the license correctly.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
PEP-621 distinguishes `authors` (people who wrote the project) from
`maintainers` (people who currently maintain it). Until now the file
only had `authors`, which obscured who is on the hook for bmaptool
today.

Add a `maintainers` block. Any tool that surfaces a project's
maintainer contact information will use it instead of falling back
to the `authors` list.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
The project keeps a curated release history at the repo root in
CHANGELOG.md. Add a `Changelog` entry to `[project.urls]` so any
tool that surfaces a project's labeled URLs can point at it
directly.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
The package version lives in the conventional location:

  src/bmaptool/__init__.py:
      __version__ = "..."

`bmaptool.CLI` re-exports it as the local `VERSION` constant
(`from . import __version__ as VERSION`), keeping the existing
`bmaptool --version` argparse wiring intact.

`[tool.hatch.version]` in pyproject.toml points at
`src/bmaptool/__init__.py`. Hatch's default regex source matches
the `__version__` line correctly and the built wheel's METADATA
carries the same version string.

`make_a_release.sh` bumps `__version__` in `src/bmaptool/__init__.py`
instead of an unrelated source file; the release flow is otherwise
unchanged.

`import bmaptool; bmaptool.__version__` now works as any Python
user expects, and the version source is no longer buried in an
argument-parsing module.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
`tests/test_compat.py` has two halves: a forward-compat check that
the current `BmapCopy` can read every historical bmap fixture in
`tests/test-data/`, and a backward-compat check that *historical*
`BmapCopy` modules under `tests/oldcodebase/` can read current
fixtures. The backward-compat half is a 2026 anachronism: a user
running bmaptool 3.x does not run BmapCopy 1.0 against today's
bmap files, and the museum modules under `tests/oldcodebase/` are
~6,500 lines of pre-Python-3 code that drags in `six` as the
only reason it stays in the dev extras.

The forward-compat half (current code, all historical fixtures)
is preserved unchanged. `tests/oldcodebase/` is removed entirely.
`tests/test_compat.py` loses `_test_older_bmapcopy()`, its inner
`import_module` helper, the `_OLDCODEBASE_SUBDIR` constant, and
the docstring line that advertised the backward-compat behavior.

`six` is removed from `[project.optional-dependencies].dev` in
pyproject.toml; no production code or surviving test imports it.

`make_a_release.sh`'s pre-release reminder
("Did you update tests: test-data and oldcodebase") drops the
`oldcodebase` clause, since there is no longer such a directory
to update.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Replace black with ruff in the dev optional-dependencies group and add
a minimal [tool.ruff] block (line-length 88, py39 target) so ruff
format is a drop-in replacement for black at the same line length.

Format-only scope on purpose: this commit does not enable
[tool.ruff.lint]. Ruff's lint side is a future, separate decision -
turning it on against the current tree surfaces a substantial backlog
across pyupgrade, bugbear, and bandit rules that deserves its own
focused pass rather than a rider on the formatter swap.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
One-time, tree-wide application of ruff format. Pure formatting churn -
no behavior changes:

- merge adjacent string literals that span continuation lines
- modernize one nested 'with' to py3.10+ parenthesized syntax
- make two trailing-comma statement expressions explicit as
  one-element tuples

This commit is isolated so it can be added to .git-blame-ignore-revs
and 'git blame' can skip it cleanly.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Provide a list of commits that 'git blame' should skip past, so that
mechanical reformatting commits never obscure the real authorship of
a line. Seeded with the single tree-wide ruff format commit.

Contributors opt in locally with:

    git config blame.ignoreRevsFile .git-blame-ignore-revs

GitHub's blame view honors this file automatically.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
The lint job still pins actions/checkout@v3, which is end-of-life
and prints a deprecation warning on every workflow run. Move it to
v4 so the warning goes away and the action keeps receiving security
updates.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Drop the psf/black@stable third-party action and run ruff format
--check . directly, so the formatter run by CI matches the [tool.ruff]
configuration in pyproject.toml.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
GitHub Actions defaults strategy.fail-fast to true, which cancels
every still-running matrix job the moment any one job fails. For a
test matrix spanning Python 3.9 through 3.14 (plus native), this
hides exactly the data point that matters most when triaging a
failure: is it specific to one interpreter, or does it reproduce
across the whole range?

Set fail-fast: false so the rest of the matrix runs to completion
and the failure pattern is visible in a single workflow run.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Floating tags like @v4 (or @main / @master) re-resolve every workflow
run. An attacker who compromises the action's repository - or simply
the maintainer's release pipeline - can have malicious code execute
inside this project's runners on the next push, without a single
commit landing here. Tag-pinning makes the action's identity part of
this repo's controlled state: the resolved commit cannot change
underneath us.

Pin actions/checkout to de0fac2e4500dabe0009e67214ff5f5447ce83dd
(v6.0.2, the current latest tag); leave the version label in a
trailing comment so the bump is human-readable and dependabot has
something to rewrite.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Floating tags re-resolve every workflow run, so a future malicious
release published under the same tag would execute inside this
project's runners on the next push, without a commit landing here.
Pinning the action by 40-char SHA freezes the action's identity into
this repo's controlled state.

Pin actions/setup-python to a309ff8b426b58ec0e2a45f0f869d46889d02405
(v6.2.0, the current latest tag); leave the version label in a
trailing comment so the bump is human-readable and dependabot has
something to rewrite.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
GitHub Actions defaults the GITHUB_TOKEN to a permissive scope (the
'read and write' setting at the repo level), which means every step
in every job inherits a token that can push commits, open issues,
modify checks, and so on. Nothing in this workflow needs any of
that; the only token use is implicit, by actions/checkout reading
the repo.

Declare permissions: contents: read at workflow scope and again on
each job. The workflow-scope block is the safety net for any future
job that forgets to set its own permissions; the per-job blocks
make each job's required surface area explicit and survive someone
later adding a job-scoped permission for a single step without
accidentally widening the default for the whole workflow.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Force pbzip2 (-p1) and pigz (-p 1) to use a single worker when the
api_base test suite is generating compressed fixtures. By default
both tools spawn one worker per detected CPU and hold a per-worker
buffer; OOM events with the default settings have occurred on
shared CI runners while compressing the test images.

Single-threaded mode is the right knob for this test:

  - pbzip2 and pigz still produce the compressed fixtures, so the
    test still exercises the producer paths.
  - The output stays a valid bzip2 / gzip stream that the
    pbzip2/pigz consumer paths in TransRead.py can still
    decompress, so consumer coverage is unchanged.
  - Peak memory during fixture generation drops; CPU concurrency
    on the producer drops; fixture-generation wall-clock goes up
    proportionally. This is a deliberate trade - finishing slower
    is acceptable; getting SIGKILLed mid-run is not.

No bmap copy behavior changes; this is a test-fixture-generation
tuning change.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Configure dependabot to open weekly pull requests against:

  - github-actions: every action used in .github/workflows/.
    Actions are SHA-pinned with '# vX.Y.Z' trailing comments,
    and dependabot rewrites both the SHA and the comment when
    a new tag ships.
  - pip: every Python package declared in pyproject.toml.
    Runtime dependencies are empty today, so this primarily
    keeps the dev extras (ruff) current.

Both ecosystems are grouped, so dependabot batches its findings
into a single PR per ecosystem per week instead of one PR per
package. That keeps the queue manageable on a low-traffic project
without giving up the security-update cadence.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Run pip-audit against the project so any known CVE in the runtime
or dev dependency graph surfaces on every push. pip-audit reads
pyproject.toml directly and queries the OSV vulnerability
database; no lockfile or project install is required.

The step is marked continue-on-error: true so a new CVE landing in
a transitive dependency does not turn the whole workflow red and
block unrelated work. Findings are visible in the run log and the
maintainer can act on them at their own cadence.

A future change can flip the step to blocking once the maintainer
is comfortable that the audit's failure modes are well-understood
(noisy false-positive CVEs, transient OSV outages, etc.).

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Add changelog entries for the metadata modernization, formatter
swap, and CI hardening:

  - Added: .git-blame-ignore-revs, .github/dependabot.yml, the
    non-blocking pip-audit step.
  - Changed: Python support floor 3.9, ceiling 3.14; ruff replaces
    black; tree-wide ruff format reformat; CI actions pinned by
    SHA; permissions: contents: read at workflow and job scope;
    fail-fast disabled on the test matrix; pbzip2/pigz pinned to
    a single worker in the api_base test to avoid OOM on shared
    runners.
  - Removed: tests/oldcodebase/ and its consumer in test_compat,
    plus six from dev extras.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
@twoerner twoerner added the ci Related to continuous integration label May 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Related to continuous integration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant