Skip reporting synthetic failures by mhdatie · Pull Request #11259 · DataDog/dd-trace-java

mhdatie · 2026-05-01T23:05:05Z

What Does This Do

Extends TagInitializationErrors.java to also tag executionError and test exception synthetic testcases
with dd_tags[test.final_status]=skip, unconditionally. Previously, only intermediate
initializationError retries were tagged.

The tagging logic is refactored into a shared tagSkip helper that uses direct child element lookups
(instead of getElementsByTagName, which searches all descendants) and is idempotent - testcases already
carrying the property are left unchanged.

This fix has been applied with Claude Code, and refactors have been made to optimize the code and enhance the comments with the help of the local code review Claude plugin. A follow up PR will be opened to rename the executor file to TagSyntheticFailures.java

Detected in CI:

Which should stop such tests from being labelled as failures (master)

Motivation

executionError and test exception are framework-level synthetic entries that never represent a real
test result. Without this change, Test Optimization treats them as genuine failures, producing noise in
the test results dashboard. Like initializationError intermediates, they should be skipped so only
real test outcomes are surfaced.

Additional Notes

Follow up PR - Rename from TagInitializationErrors.java to TagSyntheticFailures.java

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors

Jira ticket: APMLP-1296

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

github-actions · 2026-05-04T20:49:44Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

sarahchen6 · 2026-05-04T21:31:09Z

+      var name = e.getAttribute("name");
+      if ("initializationError".equals(name)) {
        byClassname.computeIfAbsent(e.getAttribute("classname"), k -> new ArrayList<>()).add(e);
+      } else if ("executionError".equals(name) || "test exception".equals(name)) {


I had codex take a look, and it pointed out that we have a real test case with the name of "test exception": https://github.com/DataDog/dd-trace-java/blob/master/dd-java-agent/instrumentation-testing/src/main/groovy/datadog/trace/agent/test/base/HttpServerTest.groovy#L1251. The test is marked as Flaky, so the output is skipped anyway IIUC, but worth noting... We could consider changing the test case name to keep this logic simpler?

good idea, will make an update here

sarahchen6 · 2026-05-04T21:33:30Z

+    return null;
+  }
+
+  static boolean tagSkip(org.w3c.dom.Document doc, Element testcase) {


Can import org.w3c.dom.Document and set this to just Document

sarahchen6

A couple small comments, but otherwise looks good!

PerfectSlayer

Updating labels and minor comments

PerfectSlayer · 2026-05-05T08:38:02Z

+    var children = parent.getChildNodes();
+    for (int i = 0; i < children.getLength(); i++) {
+      var child = children.item(i);
+      if (child instanceof Element e && tagName.equals(e.getTagName())) return e;


🔍 nitpick: Please avoid one-line for readability / uniformity (this is usually enforced using spotless on the project)‏

Suggested change

if (child instanceof Element e && tagName.equals(e.getTagName())) return e;

if (child instanceof Element e && tagName.equals(e.getTagName())) {

return e;

}

I ran ./gradlew spotlessApply and it doesn't seem to apply to .gitlab/*. When I changed the spotless.gradle config file to consider them and ran it, it wasn't applying the check here but did in other places 🤔

Claude analyzed the following:

The issue is that google-java-format does not add missing braces — it only handles whitespace, indentation, and line wrapping. Line 111: if (child instanceof Element e && tagName.equals(e.getTagName())) return e; is syntactically valid, fits within the 100-character column limit (81 chars), so the formatter has nothing to change. This is a known gap: the Google Java Style Guide §4.1.1 requires braces on all if/for/while bodies, but google-java-format deliberately does not restructure code to enforce that. You need to add the braces manually: if (child instanceof Element e && tagName.equals(e.getTagName())) { return e; }

The check to enable spotless checks on that directory was:

project.pluginManager.withPlugin('java') { java { toggleOffOn() // set explicit target to workaround https://github.com/diffplug/spotless/issues/1163 target 'src/**/*.java' // ignore embedded test projects and everything in build dir, e.g. generated sources targetExclude('src/test/resources/**', buildDirectoryFiles) tableTestFormatter('1.1.1') googleJavaFormat('1.35.0') } } // Root project doesn't apply the java plugin, so withPlugin('java') never fires for it. // Configure a separate java block to cover .gitlab/*.java scripts. if (project == project.rootProject && !project.plugins.hasPlugin('java')) { java { toggleOffOn() target '.gitlab/*.java' targetExclude(buildDirectoryFiles) tableTestFormatter('1.1.1') googleJavaFormat('1.35.0') } }

I will fix them manually for now, but wonder what your thoughts are here

PerfectSlayer · 2026-05-05T08:48:06Z

    }
-    boolean modified = false;
    for (var group : byClassname.values()) {
      if (group.size() <= 1) continue;


💭 thought: ‏Not sure what this guard is for? 🤔 The next loop should not trigger in size <= 1

heh, took me multiple attempts to convince Claude that your comment made more sense. Great catch

…lookup

…ement

dd-octo-sts · 2026-05-06T13:39:27Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-05-06T13:39:31Z

View all feedbacks in Devflow UI.

2026-05-06 13:39:30 UTC ℹ️ Start processing command /merge

2026-05-06 13:39:36 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

2026-05-06 14:49:14 UTC ℹ️ MergeQueue: This merge request was merged

mhdatie changed the title ~~Mo.atie/apmlp 1296 skip synthetic failures~~ Skip reporting synthetic failures May 1, 2026

mhdatie force-pushed the mo.atie/apmlp-1296-skip-synthetic-failures branch from 18e0884 to b93d638 Compare May 4, 2026 19:05

mhdatie marked this pull request as ready for review May 4, 2026 20:49

mhdatie requested a review from a team as a code owner May 4, 2026 20:49

mhdatie requested review from randomanderson and removed request for a team May 4, 2026 20:49

mhdatie added type: enhancement Enhancements and improvements comp: ci visibility Continuous Integration Visibility labels May 4, 2026

sarahchen6 reviewed May 4, 2026

View reviewed changes

sarahchen6 approved these changes May 4, 2026

View reviewed changes

PerfectSlayer added comp: tooling Build & Tooling and removed comp: ci visibility Continuous Integration Visibility labels May 5, 2026

PerfectSlayer approved these changes May 5, 2026

View reviewed changes

mhdatie requested a review from a team as a code owner May 5, 2026 20:53

mhdatie requested a review from ygree May 5, 2026 20:53

mhdatie added 9 commits May 5, 2026 17:46

Skip executionError and test exception synthetic testcases

fe50860

Use direct child scan instead of getElementsByTagName for properties …

63b0b2d

…lookup

Restrict property duplicate check to direct children of properties el…

f9fe71e

…ement

Improve doc example to illustrate initializationError grouping logic

abf284f

Fix inaccurate doc claims about file scope and CI behavior

0f47209

Document handled patterns and idempotency in class doc

5d6d9d0

Unify testcase loops into a single pass

1ec67de

change flaky test name

9226fc5

formatting and removing redundant check

9cb3b43

mhdatie force-pushed the mo.atie/apmlp-1296-skip-synthetic-failures branch from 49ef484 to 9cb3b43 Compare May 5, 2026 21:46

cbeauchesne approved these changes May 6, 2026

View reviewed changes

mhdatie added this pull request to the merge queue May 6, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 6, 2026

gh-worker-dd-mergequeue-cf854d Bot merged commit 7fb9855 into master May 6, 2026
570 checks passed

gh-worker-dd-mergequeue-cf854d Bot deleted the mo.atie/apmlp-1296-skip-synthetic-failures branch May 6, 2026 14:49

github-actions Bot added this to the 1.63.0 milestone May 6, 2026

Conversation

mhdatie commented May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actions Bot commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sarahchen6 May 4, 2026

Choose a reason for hiding this comment

Uh oh!

mhdatie May 5, 2026

Choose a reason for hiding this comment

Uh oh!

mhdatie May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sarahchen6 May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mhdatie May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sarahchen6 left a comment

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer May 5, 2026

Choose a reason for hiding this comment

Uh oh!

mhdatie May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mhdatie May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer May 5, 2026

Choose a reason for hiding this comment

Uh oh!

mhdatie May 5, 2026

Choose a reason for hiding this comment

Uh oh!

mhdatie May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dd-octo-sts Bot commented May 6, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mhdatie commented May 1, 2026 •

edited

Loading

github-actions Bot commented May 4, 2026 •

edited

Loading

mhdatie May 5, 2026 •

edited

Loading

sarahchen6 May 4, 2026 •

edited

Loading

mhdatie May 5, 2026 •

edited

Loading

mhdatie May 5, 2026 •

edited

Loading

mhdatie May 5, 2026 •

edited

Loading

mhdatie May 5, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented May 6, 2026 •

edited

Loading