Skip to content

feat(ci): cross-reference tag review via post-build lint + workflow_run#39666

Open
kim-em wants to merge 3 commits into
leanprover-community:masterfrom
kim-em:crossref-tooling-ci
Open

feat(ci): cross-reference tag review via post-build lint + workflow_run#39666
kim-em wants to merge 3 commits into
leanprover-community:masterfrom
kim-em:crossref-tooling-ci

Conversation

@kim-em
Copy link
Copy Markdown
Contributor

@kim-em kim-em commented May 21, 2026

Cross-reference tag review for PRs adding @[stacks ...], @[kerodon ...], or @[wikidata ...] attributes — implemented as a post-build lint step plus a workflow_run-triggered companion that posts the bot comment.

Why this shape. The previous iteration of this PR ran an independent pull_request_target workflow that text-parsed PR source files. That was a workaround for not having a built Mathlib environment available in CI. With a post-build lint step we do have the elaborated environment — so we read Mathlib.CrossRef.tagExt directly via importModules (loadExts := true) and cut ~250 LOC of bespoke byte-level scanner.

The three pieces:

  • scripts/crossref.lean — the extract subcommand is gone. New check --diff <range> --tsv <path> subcommand walks tagExt, looks up each tagged declaration's source module via env.getModuleFor?, filters by git diff --name-only of the PR range, and writes a machine-readable TSV. No Markdown, no network, no snippet fetching — that's the privileged workflow's job. Note: withImportModules passes loadExts := false and would leave tagExt empty for imported modules; we use importModules (loadExts := true) instead.

  • .github/workflows/build_template.yml — three new steps in post_steps, after the existing import-graph upload:

    1. lake env lean --run scripts/crossref.lean check --diff "${BASE_SHA}..HEAD" --tsv crossref-added.tsv
    2. Write crossref-context.json carrying the PR number and head SHA (so the workflow_run job doesn't have to rely on the unreliable workflow_run.pull_requests[0].number).
    3. Upload both as a single crossref-context artifact.
  • .github/workflows/crossref_review.yml — rewritten. Trigger flips from pull_request_target to workflow_run on the build workflows. Downloads the artifact, resolves the PR number through a three-stage fallback (artifact JSON → workflow_run payload → gh pr list head SHA match), calls the mathlib-ci orchestrator (which now consumes the TSV and fetches snippets in Python directly with parallel ThreadPoolExecutor for Stacks/Kerodon), and posts via update_PR_comment.sh. Fails the check on any missing tag.

Privilege model. The lint step runs with the PR's permissions and produces only a machine-readable TSV. The workflow_run job runs in the base ref's permission scope (with pull-requests: write), never executes anything from the artifact, fetches snippets server-side from three fixed upstream APIs, and builds the Markdown comment in trusted code.

🤖 Prepared with Claude Code

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 21, 2026

PR summary 292d3968f8

Import changes for modified files

No significant changes to the import graph

Import changes for all files
Files Import difference
Mathlib.Tactic 2
Mathlib.Tactic.CrossRef.Fetch (new file) 54

Declarations diff

+ CheckArgs
+ CrossRefHoverPanel
+ CrossRefHoverPanel.rpc
+ CrossRefHoverProps
+ Database.fetch
+ Database.gerbyBase?
+ Database.ofName?
+ Database.ofName?_name
+ Database.ofString?
+ Database.ofString?_name
+ Result
+ Result.emit
+ SnippetOutcome
+ WikidataParse
+ attachCrossRefWidget
+ cacheFile?
+ cacheLoad
+ cacheStore
+ checkUsage
+ fetchGerbyAll
+ fetchSnippet
+ gerbyBase?
+ gitChangedFiles
+ moduleToFilePath
+ parseCheckArgs
+ parseWikidataResponse
+ renderOutcome
+ snippetMain
+ snippetUsage
+ usage
+ wikidataBatchSize
+ wikidataResultOf
++ Database.name
++ afterFirst?
++ fetchGerby
++ fetchUrl
++ fetchWikidata
++ flattenWhitespace
++ jsonStrPath?
++ parseGerbyTitle
++ stripHtml
++ takeUntilChar
++ userAgent
+-+ Database

You can run this locally as follows
## from your `mathlib4` directory:
git clone https://github.com/leanprover-community/mathlib-ci.git ../mathlib-ci

## summary with just the declaration names:
../mathlib-ci/scripts/pr_summary/declarations_diff.sh <optional_commit>

## more verbose report:
../mathlib-ci/scripts/pr_summary/declarations_diff.sh long <optional_commit>

The doc-module for scripts/pr_summary/declarations_diff.sh in the mathlib-ci repository contains some details about this script.


No changes to strong technical debt.
No changes to weak technical debt.

⚠️ Workflow documentation reminder

This PR modifies files under .github/workflows/.
Please update docs/workflows.md if the workflow inventory, triggers, or behavior changed.

Modified workflow files:

  • .github/workflows/build_template.yml
  • .github/workflows/crossref_review.yml

⚠️ Scripts folder reminder

This PR adds files under scripts/.
Please consider whether each added script belongs in this repository or in leanprover-community/mathlib-ci.

A script belongs in mathlib-ci if it is a CI automation script that interacts with GitHub (e.g. managing labels, posting comments, triggering bots), runs from a trusted external checkout in CI, or requires access to secrets.

A script belongs in this repository (scripts/) if it is a developer or maintainer tool to be run locally, a code maintenance or analysis utility, a style linting tool, or a data file used by the library's own linters.

See the mathlib-ci README for more details.

Added scripts files:

  • scripts/crossref.lean

@mathlib-dependent-issues mathlib-dependent-issues Bot added the blocked-by-other-PR This PR depends on another PR (this label is automatically managed by a bot) label May 21, 2026
@mathlib-dependent-issues
Copy link
Copy Markdown

mathlib-dependent-issues Bot commented May 21, 2026

@kim-em
Copy link
Copy Markdown
Contributor Author

kim-em commented May 22, 2026

Here's what the bot's comment renders as, generated by feeding the orchestrator a synthetic file with one valid + one bogus tag from each database. (No quoting trick — this is the literal contents of comment.md from a real run, dropped into a GitHub comment so you can see how the table actually displays.)


Cross-reference tags added by this PR

Wikidata

Tag Declaration Title Snippet Author comment
Q11518 theorem pythagoras : True Pythagorean theorem relation in Euclidean geometry among the three sides of a right triangle named after Pythagoras
QFAKE99999 theorem bad : True NOT FOUND intentionally bogus

Stacks

Tag Declaration Title Snippet Author comment
09FS theorem cohen_first (n : ℕ) [NeZero n] : True Example 9.5.2 Example 9.5.2. The characteristic of $\mathbf{F}_ p$ is $p$, and that of $\mathbf{Q}$ is $0$. First part. We don't require p to be a prime in mathlib.
ZZZZ theorem worse : True NOT FOUND also bogus

Kerodon

Tag Declaration Title Snippet Author comment
0009 theorem first_notation : True Notation 1.1.0.1 Notation 1.1.0.1. For every nonnegative integer $n$, we let $[n]$ denote the linearly ordered set ${ 0 &lt; 1 &lt; 2 &lt; \cdots &lt; n-1 &lt; n } $.

⚠ One or more tags above could not be found upstream. Please verify the tag identifier or remove the attribute.

@kim-em kim-em force-pushed the crossref-tooling-ci branch 4 times, most recently from 69c93c6 to 1eb21ea Compare May 22, 2026 10:05
kim-em and others added 2 commits May 22, 2026 10:13
Add `scripts/crossref.lean`, a single standalone tool to support review of
PRs that add `@[stacks ...]`, `@[kerodon ...]`, or `@[wikidata ...]`
attributes. It exposes two subcommands:

* `lake env lean --run scripts/crossref.lean snippet <db> <tag>...` fetches
  a one-line label/description for each tag from the upstream database.
  Wikidata uses the `wbgetentities` API (with recovery when one bad QID
  poisons a batch); Stacks and Kerodon use the documented Gerby
  `/data/tag/<TAG>/content/statement` endpoint with a tolerant HTML→text
  strip that survives `<` inside math. Exit codes distinguish all-resolved
  (0) from missing (2) from transient network failure (3). When
  `CROSSREF_CACHE_DIR` is set, responses are memoised per `(database, tag)`
  so repeat lookups are instant.

* `lake env lean --run scripts/crossref.lean extract --file <path>...` (or
  `--diff <range>`) walks Lean source and emits TSV of every
  cross-reference attribute it finds, paired with the declaration it
  decorates. A byte-level scanner correctly skips string literals and
  line/block comments, handles multi-attribute blocks
  (`@[simp, stacks 01AB]`), multi-line attribute blocks, doc comments
  between attribute and declaration, and modifier keywords (`private`,
  `noncomputable`, …). Signatures are collapsed to one line for downstream
  consumption.

The two subcommands share a single `Database` enum, with a roundtrip
theorem `Database.ofString?_name` proving `name ∘ ofString?` is a section
— so adding a fourth database fails to compile until both projections are
updated together.

The script imports only Lean core (plus `Std.Data.HashMap` for batching),
so it can be invoked directly with `lake env lean --run` without any
Mathlib build, and a future CI workflow can drive it on untrusted PR
files without elaborating PR code.

Companion follow-ups (separate PRs) will add an LSP widget that surfaces
these snippets in the editor and a GitHub Actions workflow that posts a
once-per-PR comment with tag / signature / snippet tables and fails CI on
unresolved tags.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a Mathlib declaration carries `@[stacks ...]`, `@[kerodon ...]`, or
`@[wikidata ...]`, the info view now shows the upstream label / description
when the cursor sits on the attribute. The fetch is on-demand via an RPC
call from the widget to the Lean server, which shells out to `curl`; we
deliberately don't fetch at attribute-elaboration time so offline builds
stay clean and the only network access is when an editing session opens
the panel.

Two new modules and a small refactor to the existing one:

* `Mathlib/Tactic/CrossRef/Fetch.lean` (new) holds the `Database` enum and
  the snippet-fetch logic. The companion `scripts/crossref-snippet.lean`
  in the previous PR reimplements the same logic standalone for CI usage;
  the two are kept in sync deliberately so the CI script doesn't need a
  Mathlib build.
* `Mathlib/Tactic/Widget/CrossRefHover.lean` (new) defines the widget
  itself: an `RpcEncodable` props record, a `RequestM` RPC method that
  calls `fetchSnippet`, and a `mk_rpc_widget%` Component that renders
  the result as Html.
* `Mathlib/Tactic/CrossRefAttribute.lean` (modified) imports the two new
  modules, removes the now-duplicated `Database` / `databaseURL` /
  `databaseLabel` definitions, and calls `Widget.savePanelWidgetInfo` in
  each attribute's `add` handler. Tag storage, docstring rewriting, and
  the trace commands are untouched.

If `curl` is missing or the upstream site is unreachable, the widget
renders an inline error instead of blocking the LSP. The existing
`MathlibTest/CrossRefAttribute.lean` suite continues to pass — none of
the behaviour visible to it changes.

Stacked on leanprover-community#39662
(scripts only). A third follow-up will add a GitHub Actions workflow
that posts a once-per-PR comment with tag / signature / snippet tables.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@kim-em kim-em force-pushed the crossref-tooling-ci branch from 1eb21ea to 9d70518 Compare May 22, 2026 10:17
kim-em added a commit to kim-em/mathlib-ci that referenced this pull request May 22, 2026
…t step

The mathlib4 cross-reference review pipeline is moving from a text-based
extractor running independently in a `pull_request_target` workflow to an
elaboration-based lint step that runs in the build pipeline's `post_steps`
job after `lake exe cache get`. The lint step emits a machine-readable
TSV (db / tag / declName / file / comment) as an artifact, and a
`workflow_run`-triggered companion downloads it here.

This commit rewrites `scripts/crossref_review/crossref-pr-comment.py` so
it:

* Takes `--tsv` (the path to the artifact) instead of two checkout paths.
* Defensively parses the TSV (unknown databases dropped, missing fields
  padded). Nothing from PR-derived input is executed.
* Fetches snippets directly via Python's `urllib` — Wikidata via the
  batched `wbgetentities` API, Stacks/Kerodon via the documented Gerby
  `/data/tag/<TAG>/content/statement` endpoint. Gerby fetches are
  parallelised with `ThreadPoolExecutor(max_workers=8)` to keep the
  workflow under a minute on PRs that touch large files.
* Constructs the Markdown comment in trusted code (no longer relies on
  PR-derived Markdown), still using the `### Cross-reference tags added
  by this PR` sentinel so `update_PR_comment.sh` can dedupe.
* Exits 0 / 1 / 2 / 3 with the same grammar as before.

Stacked on leanprover-community/mathlib4#39666
upcoming force-push, which adds the post-build lint step and rewires
`crossref_review.yml` as `workflow_run`-triggered.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@kim-em kim-em force-pushed the crossref-tooling-ci branch from 9d70518 to d49feaa Compare May 22, 2026 11:39
@kim-em kim-em changed the title feat(ci): per-PR comment summarising cross-reference tags feat(ci): cross-reference tag review via post-build lint + workflow_run May 22, 2026
Replace the text-based cross-reference extractor with an elaboration-
based lint step that runs in the build pipeline's `post_steps` job
and a `workflow_run`-triggered companion that posts the bot comment.

**Why.** The previous text-based extractor (a ~250 LOC byte-level
scanner in `scripts/crossref.lean`) was a workaround for not having a
built Mathlib environment in the pull_request_target workflow. Now
that we run as a post-build lint step, we have a fully elaborated
environment and can read `Mathlib.CrossRef.tagExt` directly — cutting
~250 LOC and eliminating the fidelity gap with the Lean parser.

**Pieces:**

* `scripts/crossref.lean` — the `extract` subcommand is gone. New
  `check --diff <range> --tsv <path>` subcommand uses
  `importModules (loadExts := true)` (NOT `withImportModules`, which
  passes `loadExts := false` and would leave `tagExt` empty), walks
  `tagExt`, looks up each tagged declaration's source module via
  `env.getModuleFor?`, and filters by `git diff --name-only` of the
  PR range. Output is machine-readable TSV
  (`db\ttag\tdeclName\tfile\tcomment`). No Markdown, no network,
  no snippets — those are the privileged workflow's job.

* `.github/workflows/build_template.yml` — adds three new steps to
  `post_steps`, after the import-graph upload:
  - Run `lake env lean --run scripts/crossref.lean check ...`
    against the diff between the PR base and HEAD.
  - Write a `crossref-context.json` with PR number + head SHA so
    the companion workflow doesn't have to rely on the unreliable
    `workflow_run.pull_requests[0].number`.
  - Upload both as a single `crossref-context` artifact.

* `.github/workflows/crossref_review.yml` — rewritten. Old trigger
  `pull_request_target` → new trigger `workflow_run`. Downloads the
  artifact, resolves the PR number through a three-stage fallback
  chain (artifact JSON → workflow_run payload → `gh pr list` head
  SHA match), calls the mathlib-ci orchestrator (which now consumes
  the TSV and fetches snippets in Python), posts/updates/deletes
  via `update_PR_comment.sh`, and fails the check on `missing` tags.

**Privilege model.** The lint step runs with the PR's permissions and
produces only machine-readable TSV. The `workflow_run` job runs in the
base ref's permission scope (with `pull-requests: write`) and never
executes anything from the artifact — it parses the TSV defensively,
fetches snippets server-side from three fixed upstream APIs, and
builds the Markdown comment in trusted code.

Companion mathlib-ci PR:
leanprover-community/mathlib-ci#39

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@kim-em kim-em force-pushed the crossref-tooling-ci branch from d49feaa to 292d396 Compare May 22, 2026 12:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blocked-by-other-PR This PR depends on another PR (this label is automatically managed by a bot)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant