Build Failure BatchTestLocal #6674
Description
Expected behavior
All tests should run without failures.
Actual behavior
At least in https://github.com/apache/jmeter/actions/runs/23376587233/job/68009514670 on MacOS with JDK 17 we got an error in BatchTestLocal.
It seems, that there are more active thread groups and threads than wanted. The test assumes working with consecutive thread groups with each one thread. So there should never be a second thread group or thread visible.
So either we are counting wrong, or we "lost" a thread thread group from an earlier consecutive group.
> Task :src:dist-check:batchServerBatchTestLocal
summary + 131 in 00:00:16 = 8,2/s Avg: 27 Min: 0 Max: 199 Err: 16 (12,21%) Active: 0 Started: 11 Finished: 11
summary = 132 in 00:00:17 = 8,0/s Avg: 27 Min: 0 Max: 199 Err: 16 (12,12%)
Tidying up remote @ 2026 Mar 21 09:49:18 UTC (1774086558613)
... end of run
ERROR: unexpected output for BatchTestLocal.csv:
- expected 6457 bytes, /Users/runner/work/jmeter/jmeter/bin/testfiles/BatchTestLocal.csv
+ actual 6457 bytes, /Users/runner/work/jmeter/jmeter/bin/BatchTestLocal.csv
@@ -48,11 +48,11 @@
Java If once 1,,,text,false,,0,1,1,null,,1,1
Java If once 2,200,OK,text,true,,0,1,1,null,,1,0
Java If all 1,,,text,false,,0,1,1,null,,1,1
-Java OK,200,OK,text,true,,0,1,1,null,,1,0
-Java If once 1,,,text,false,,0,1,1,null,,1,1
-Java If once 2,200,OK,text,true,,0,1,1,null,,1,0
-Java If all 1,,,text,false,,0,1,1,null,,1,1
-Java OK,200,OK,text,true,,0,1,1,null,,1,0
+Java OK,200,OK,text,true,,0,2,2,null,,1,0
+Java If once 1,,,text,false,,0,2,2,null,,1,1
+Java If once 2,200,OK,text,true,,0,2,2,null,,1,0
+Java If all 1,,,text,false,,0,2,2,null,,1,1
+Java OK,200,OK,text,true,,0,2,2,null,,1,0
Java 1 C1=1,200,OK,text,true,,10,1,1,null,,1,0
Java 1 C1=1,200,OK,text,true,,10,1,1,null,,1,0
Java 1 C1=1,200,OK,text,true,,10,1,1,null,,1,0
ERROR: unexpected output for BatchTestLocal.xml:
- expected 37708 bytes, /Users/runner/work/jmeter/jmeter/bin/testfiles/BatchTestLocal.xml
+ actual 37708 bytes, /Users/runner/work/jmeter/jmeter/bin/BatchTestLocal.xml
@@ -299,31 +299,31 @@
<responseData class="java.lang.String"/>
<responseFile class="java.lang.String"/>
</sample>
-<sample s="true" lb="Java OK" rc="200" rm="OK" dt="text" by="0" sc="1" ec="0" ng="1" na="1">
+<sample s="true" lb="Java OK" rc="200" rm="OK" dt="text" by="0" sc="1" ec="0" ng="2" na="2">
<responseHeader class="java.lang.String"/>
<requestHeader class="java.lang.String"/>
<responseData class="java.lang.String"/>
<responseFile class="java.lang.String"/>
</sample>
-<sample s="false" lb="Java If once 1" rc="" rm="" dt="text" by="0" sc="1" ec="1" ng="1" na="1">
+<sample s="false" lb="Java If once 1" rc="" rm="" dt="text" by="0" sc="1" ec="1" ng="2" na="2">
<responseHeader class="java.lang.String"/>
<requestHeader class="java.lang.String"/>
<responseData class="java.lang.String"/>
<responseFile class="java.lang.String"/>
</sample>
-<sample s="true" lb="Java If once 2" rc="200" rm="OK" dt="text" by="0" sc="1" ec="0" ng="1" na="1">
+<sample s="true" lb="Java If once 2" rc="200" rm="OK" dt="text" by="0" sc="1" ec="0" ng="2" na="2">
<responseHeader class="java.lang.String"/>
<requestHeader class="java.lang.String"/>
<responseData class="java.lang.String"/>
<responseFile class="java.lang.String"/>
</sample>
-<sample s="false" lb="Java If all 1" rc="" rm="" dt="text" by="0" sc="1" ec="1" ng="1" na="1">
+<sample s="false" lb="Java If all 1" rc="" rm="" dt="text" by="0" sc="1" ec="1" ng="2" na="2">
<responseHeader class="java.lang.String"/>
<requestHeader class="java.lang.String"/>
<responseData class="java.lang.String"/>
<responseFile class="java.lang.String"/>
</sample>
-<sample s="true" lb="Java OK" rc="200" rm="OK" dt="text" by="0" sc="1" ec="0" ng="1" na="1">
+<sample s="true" lb="Java OK" rc="200" rm="OK" dt="text" by="0" sc="1" ec="0" ng="2" na="2">
<responseHeader class="java.lang.String"/>
<requestHeader class="java.lang.String"/>
<responseData class="java.lang.String"/>
No errors present in the logfile /Users/runner/work/jmeter/jmeter/bin/BatchTestLocal.log (the file is empty)
No errors present in the logfile /Users/runner/work/jmeter/jmeter/bin/BatchTestLocalServer.log (the file is empty)
> Task :src:dist-check:batchServerBatchTestLocal FAILED
:src:dist-check:batchServerBatchTestLocal failure marker
:src:dist-check:batchServerBatchTestLocal failure marker
Build FAILURE reason:
Execution failed for task ':src:dist-check:batchServerBatchTestLocal':
Failures detected while testing /Users/runner/work/jmeter/jmeter/bin/testfiles/BatchTestLocal.jmx: [unexpected output BatchTestLocal.csv, unexpected output BatchTestLocal.xml]
at org.apache.jmeter.buildtools.batchtest.BatchTest.exec(BatchTest.kt:226)
at org.apache.jmeter.buildtools.batchtest.BatchTestServer.exec(BatchTestServer.kt:134)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:125)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:58)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29)
at org.gradle.api.internal.tasks.execution.TaskExecution$3.run(TaskExecution.java:252)
at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:166)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47)
at org.gradle.api.internal.tasks.execution.TaskExecution.executeAction(TaskExecution.java:237)
at org.gradle.api.internal.tasks.execution.TaskExecution.executeActions(TaskExecution.java:220)
at org.gradle.api.internal.tasks.execution.TaskExecution.executeWithPreviousOutputFiles(TaskExecution.java:203)
at org.gradle.api.internal.tasks.execution.TaskExecution.execute(TaskExecution.java:170)
at org.gradle.internal.execution.steps.ExecuteStep.executeInternal(ExecuteStep.java:105)
at org.gradle.internal.execution.steps.ExecuteStep.access$000(ExecuteStep.java:44)
at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:59)
at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:56)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:209)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:166)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:56)
at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:44)
at org.gradle.internal.execution.steps.CancelExecutionStep.execute(CancelExecutionStep.java:42)
at org.gradle.internal.execution.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:75)
at org.gradle.internal.execution.steps.TimeoutStep.execute(TimeoutStep.java:55)
at org.gradle.internal.execution.steps.PreCreateOutputParentsStep.execute(PreCreateOutputParentsStep.java:50)
at org.gradle.internal.execution.steps.PreCreateOutputParentsStep.execute(PreCreateOutputParentsStep.java:28)
at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:68)
at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:38)
at org.gradle.internal.execution.steps.BroadcastChangingOutputsStep.execute(BroadcastChangingOutputsStep.java:61)
at org.gradle.internal.execution.steps.BroadcastChangingOutputsStep.execute(BroadcastChangingOutputsStep.java:26)
at org.gradle.internal.execution.steps.CaptureOutputsAfterExecutionStep.execute(CaptureOutputsAfterExecutionStep.java:69)
at org.gradle.internal.execution.steps.CaptureOutputsAfterExecutionStep.execute(CaptureOutputsAfterExecutionStep.java:46)
at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:39)
at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:28)
at org.gradle.internal.execution.steps.BuildCacheStep.executeWithoutCache(BuildCacheStep.java:189)
at org.gradle.internal.execution.steps.BuildCacheStep.lambda$execute$1(BuildCacheStep.java:75)
at org.gradle.internal.Either$Right.fold(Either.java:176)
at org.gradle.internal.execution.caching.CachingState.fold(CachingState.java:62)
at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:73)
at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:48)
at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:46)
at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:35)
at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:75)
at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$2(SkipUpToDateStep.java:53)
at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:53)
at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:35)
at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:37)
at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:27)
at org.gradle.internal.execution.steps.ResolveIncrementalCachingStateStep.executeDelegate(ResolveIncrementalCachingStateStep.java:49)
at org.gradle.internal.execution.steps.ResolveIncrementalCachingStateStep.executeDelegate(ResolveIncrementalCachingStateStep.java:27)
at org.gradle.internal.execution.steps.AbstractResolveCachingStateStep.execute(AbstractResolveCachingStateStep.java:71)
at org.gradle.internal.execution.steps.AbstractResolveCachingStateStep.execute(AbstractResolveCachingStateStep.java:39)
at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:64)
at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:35)
at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:62)
at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:40)
at org.gradle.internal.execution.steps.AbstractCaptureStateBeforeExecutionStep.execute(AbstractCaptureStateBeforeExecutionStep.java:76)
at org.gradle.internal.execution.steps.AbstractCaptureStateBeforeExecutionStep.execute(AbstractCaptureStateBeforeExecutionStep.java:45)
at org.gradle.internal.execution.steps.AbstractSkipEmptyWorkStep.executeWithNonEmptySources(AbstractSkipEmptyWorkStep.java:136)
at org.gradle.internal.execution.steps.AbstractSkipEmptyWorkStep.execute(AbstractSkipEmptyWorkStep.java:61)
at org.gradle.internal.execution.steps.AbstractSkipEmptyWorkStep.execute(AbstractSkipEmptyWorkStep.java:38)
at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:38)
at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:36)
at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:23)
at org.gradle.internal.execution.steps.HandleStaleOutputsStep.execute(HandleStaleOutputsStep.java:75)
at org.gradle.internal.execution.steps.HandleStaleOutputsStep.execute(HandleStaleOutputsStep.java:41)
at org.gradle.internal.execution.steps.AssignMutableWorkspaceStep.lambda$execute$0(AssignMutableWorkspaceStep.java:35)
at org.gradle.api.internal.tasks.execution.TaskExecution$4.withWorkspace(TaskExecution.java:297)
at org.gradle.internal.execution.steps.AssignMutableWorkspaceStep.execute(AssignMutableWorkspaceStep.java:31)
at org.gradle.internal.execution.steps.AssignMutableWorkspaceStep.execute(AssignMutableWorkspaceStep.java:22)
at org.gradle.internal.execution.steps.ChoosePipelineStep.execute(ChoosePipelineStep.java:40)
at org.gradle.internal.execution.steps.ChoosePipelineStep.execute(ChoosePipelineStep.java:23)
at org.gradle.internal.execution.steps.ExecuteWorkBuildOperationFiringStep.lambda$execute$2(ExecuteWorkBuildOperationFiringStep.java:67)
at org.gradle.internal.execution.steps.ExecuteWorkBuildOperationFiringStep.execute(ExecuteWorkBuildOperationFiringStep.java:67)
at org.gradle.internal.execution.steps.ExecuteWorkBuildOperationFiringStep.execute(ExecuteWorkBuildOperationFiringStep.java:39)
at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:46)
at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:34)
at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:44)
at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:31)
[Incubating] Problems report is available at: file:///Users/runner/work/jmeter/jmeter/build/reports/problems/problems-report.html
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':src:dist-check:batchServerBatchTestLocal'.
> Failures detected while testing /Users/runner/work/jmeter/jmeter/bin/testfiles/BatchTestLocal.jmx: [unexpected output BatchTestLocal.csv, unexpected output BatchTestLocal.xml]
Steps to reproduce the problem
Not sure yet, either we need a Mac (under load?) or run the test many times. It seems to be a flaky test, as it doesn't happen every time.
JMeter Version
current main branch
Java Version
No response
OS Version
No response