Use post-processing approach for test report formatting to support Gradle 9 #891

FinlayRJW · 2026-01-23T14:44:49Z

Before this PR

The TestReportFormattingPlugin relied on internal Gradle APIs (HtmlTestReport, TestReporter, etc.) using reflection to intercept and format test report output. This approach broke frequently across Gradle versions, requiring version-specific reflection hacks for 8.8, 8.11, and now completely
breaking in 9.3.0 where org/gradle/api/internal/tasks/testing/report/HtmlTestReport was removed entirely.

After this PR

Replaces the reflection-heavy internal API approach with a simple post-processing strategy. The plugin now uses FlowScope.always() to run after test tasks complete, walking the generated HTML report directory and reformatting witchcraft log JSON lines in-place. This approach works across all Gradle 8.1+
versions including 9.x without version-specific code

==COMMIT_MSG==
Use post-processing approach for test report formatting to support Gradle 9
==COMMIT_MSG==

Possible downsides?

Drops Gradle 7 support. The post-processing happens after test execution completes rather than during report generation, but the end result is equivalent.

changelog-app · 2026-01-23T14:44:54Z

Generate changelog in `changelog/@unreleased`

Type (Select exactly one)

Feature (Adding new functionality)
Improvement (Improving existing functionality)
Fix (Fixing an issue with existing functionality)
Break (Creating a new major version by breaking public APIs)
Deprecation (Removing functionality in a non-breaking way)
Migration (Automatically moving data/functionality to a new system)

Description

Use post-processing approach for test report formatting to support Gradle 9

Check the box to generate changelog(s)

Generate changelog entry

...r/witchcraft/java/logging/gradle/testreport/TestReportFormattingPluginIntegrationSpec.groovy

...n/groovy/com/palantir/witchcraft/java/logging/gradle/testreport/HtmlReportPostProcessor.java

…h2>...<pre>...</pre>

changelog-app · 2026-01-26T14:17:24Z

✅ Successfully generated changelog entry!

Need to regenerate?

Simply interact with the changelog bot comment again to regenerate these entries.

🔄 Changelog entries were re-generated at Mon, 26 Jan 2026 14:20:19 UTC!

📋Changelog Preview

💡 Improvements

Use post-processing approach for test report formatting to support Gradle 9 (#891)

...n/groovy/com/palantir/witchcraft/java/logging/gradle/testreport/HtmlReportPostProcessor.java

felixdesouza · 2026-01-26T14:25:21Z

...roovy/com/palantir/witchcraft/java/logging/gradle/testreport/TestReportFormattingPlugin.java

-            }
+            getFlowScope().always(FormatReportAction.class, spec -> spec.getParameters()
+                    .getReportDir()
+                    .set(task.getReports().getHtml()));


isn't this just a last write wins thing? or am I misunderstanding

ah, always creates a new thing each time

CRogers · 2026-01-26T14:26:14Z

...tir/witchcraft/java/logging/gradle/testreport/TestReportFormattingPluginIntegrationTest.java

+        // Gradle 8 uses classes/com.palantir.SimpleTest.html
+        // Gradle 9 uses com.palantir.SimpleTest/simpleTest.html


This is maybe one place where it might be nice for the testing framework to allowing injecting a GradleVersion into the test.

Perhaps overkill, but bonus points if there's some way of calling a method like:

gradleVersion.ifLessThanOrElse( GradleVerison.of("9.0.0"), "reports/tests/test/classes/com.palantir.SimpleTest.html", "reports/tests/test/com.palantir.SimpleTest/simpleTest.html")

...that could be automatically changed by an errorprone to just:

"reports/tests/test/com.palantir.SimpleTest/simpleTest.html"

...if we remove Gradle 8 support.

oh I missed your comment

I was going to suggest some sort of parameterised test, that had a version parameter, and you could potentially do Assumptions.assumeThat(...).

I think you could probably even do it without injecting the GradleVersion. Just doing the assumption call during injection or something or class creation.

...roovy/com/palantir/witchcraft/java/logging/gradle/testreport/TestReportFormattingPlugin.java

felixdesouza · 2026-01-26T14:29:44Z

...tir/witchcraft/java/logging/gradle/testreport/TestReportFormattingPluginIntegrationTest.java

+        // Gradle 8 uses classes/com.palantir.SimpleTest.html
+        // Gradle 9 uses com.palantir.SimpleTest/simpleTest.html
+        ArbitraryFile legacyReportFile =
+                rootProject.buildDir().file("reports/tests/test/classes/com.palantir.SimpleTest.html");
+        ArbitraryFile newReportFile =
+                rootProject.buildDir().file("reports/tests/test/com.palantir.SimpleTest/simpleTest.html");


I feel like this is a little weird, can we not pass it in as a parameter or something? not sure how well it fits into the framework @CRogers what do you think?

I think we could do something quite sensible with @RestrictToGradleVersionsEqualTo if we can get that in which would let us make a version of the test just for gradle 8.14.3

that still requires you to make two separate tests even though the content is identical

see my comment here: https://github.com/palantir/witchcraft-java-logging/pull/891/changes#r2727818546

...n/groovy/com/palantir/witchcraft/java/logging/gradle/testreport/HtmlReportPostProcessor.java

felixdesouza · 2026-01-26T15:13:59Z

...n/groovy/com/palantir/witchcraft/java/logging/gradle/testreport/HtmlReportPostProcessor.java

+
+    // Match <pre> tags within stdout/stderr sections: <h2>standard output</h2>...<pre>...</pre>
+    private static final Pattern OUTPUT_SECTION_PATTERN = Pattern.compile(
+            "(<h2>standard (?:output|error)</h2>\\s*<span[^>]*>\\s*<pre[^>]*>)(.*?)(</pre>)",


one thing to note is whether or not this regex is greedy, which I think you want

I think the current is correct no? The .*? is non-greedy, which is correct here. It stops at the first </pre> after each opening tag, so multiple stdout/stderr sections are matched individually rather than being consumed as one giant match

maybe rubbish

59fb8b7

probot-autolabeler bot added the autorelease label Jan 23, 2026

felixdesouza reviewed Jan 23, 2026

View reviewed changes

...r/witchcraft/java/logging/gradle/testreport/TestReportFormattingPluginIntegrationSpec.groovy Outdated Show resolved Hide resolved

felixdesouza reviewed Jan 23, 2026

View reviewed changes

...n/groovy/com/palantir/witchcraft/java/logging/gradle/testreport/HtmlReportPostProcessor.java Outdated Show resolved Hide resolved

FinlayRJW added 4 commits January 23, 2026 14:51

spotless and the like

259ca9d

match <pre> tags within stdout/stderr sections: <h2>standard output</…

97ad71a

…h2>...<pre>...</pre>

parity match

5cb6362

merge

f88524a

FinlayRJW changed the title ~~maybe rubbish~~ Use post-processing approach for test report formatting to support Gradle 9 Jan 23, 2026

FinlayRJW added 2 commits January 23, 2026 17:01

spotless

780431b

tidier

7f38c02

FinlayRJW marked this pull request as ready for review January 26, 2026 14:16

CRogers reviewed Jan 26, 2026

View reviewed changes

...n/groovy/com/palantir/witchcraft/java/logging/gradle/testreport/HtmlReportPostProcessor.java Show resolved Hide resolved

felixdesouza reviewed Jan 26, 2026

View reviewed changes

...n/groovy/com/palantir/witchcraft/java/logging/gradle/testreport/HtmlReportPostProcessor.java Outdated Show resolved Hide resolved

felixdesouza reviewed Jan 26, 2026

View reviewed changes

CRogers reviewed Jan 26, 2026

View reviewed changes

...roovy/com/palantir/witchcraft/java/logging/gradle/testreport/TestReportFormattingPlugin.java Show resolved Hide resolved

felixdesouza reviewed Jan 26, 2026

View reviewed changes

...roovy/com/palantir/witchcraft/java/logging/gradle/testreport/TestReportFormattingPlugin.java Outdated Show resolved Hide resolved

felixdesouza reviewed Jan 26, 2026

View reviewed changes

check gradle version

21f76f1

felixdesouza reviewed Jan 26, 2026

View reviewed changes

...n/groovy/com/palantir/witchcraft/java/logging/gradle/testreport/HtmlReportPostProcessor.java Outdated Show resolved Hide resolved

FinlayRJW added 5 commits January 26, 2026 14:41

optional

7ac0be9

fixup

4b0c1b7

make utils class

968c07d

spotless

fea4d1a

whoops

a60fd3a

felixdesouza reviewed Jan 26, 2026

View reviewed changes

...n/groovy/com/palantir/witchcraft/java/logging/gradle/testreport/HtmlReportPostProcessor.java Outdated Show resolved Hide resolved

felixdesouza reviewed Jan 26, 2026

View reviewed changes

dont handroll

940ef4b

		// Gradle 8 uses classes/com.palantir.SimpleTest.html
		// Gradle 9 uses com.palantir.SimpleTest/simpleTest.html

Use post-processing approach for test report formatting to support Gradle 9 #891

Are you sure you want to change the base?

Use post-processing approach for test report formatting to support Gradle 9 #891

Uh oh!

Conversation

FinlayRJW commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Before this PR

After this PR

Possible downsides?

Uh oh!

changelog-app bot commented Jan 23, 2026 • edited by FinlayRJW Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Generate changelog in changelog/@unreleased

Uh oh!

Uh oh!

Uh oh!

changelog-app bot commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Successfully generated changelog entry!

Need to regenerate?

🔄 Changelog entries were re-generated at Mon, 26 Jan 2026 14:20:19 UTC!

📋Changelog Preview

💡 Improvements

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CRogers Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

FinlayRJW commented Jan 23, 2026 •

edited

Loading

changelog-app bot commented Jan 23, 2026 •

edited by FinlayRJW

Loading

Generate changelog in `changelog/@unreleased`

changelog-app bot commented Jan 26, 2026 •

edited

Loading

CRogers Jan 26, 2026 •

edited

Loading