Add apm-sdks-benchmarks to CI by sarahchen6 · Pull Request #10775 · DataDog/dd-trace-java

sarahchen6 · 2026-03-09T17:42:02Z

What Does This Do

Add apm-sdks-benchmarks to CI

Motivation

This is a precursor to #10550 that will format these benchmarks and replace the dd-trace-java local ones

Additional Notes

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors

Jira ticket: https://datadoghq.atlassian.net/browse/APMLP-714

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

pr-commenter · 2026-03-09T18:24:41Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/add-benchmarks
git_commit_date	1773169596	1773171313
git_commit_sha	`cc12228`	`a03549e`
release_version	1.61.0-SNAPSHOT~cc122288e5	1.61.0-SNAPSHOT~a03549e915

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1773173100	1773173100
ci_job_id	1494355677	1494355677
ci_pipeline_id	101726144	101726144
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-dpdi8e0w 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-dpdi8e0w 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 64 metrics, 7 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~a03549e915, baseline=1.61.0-SNAPSHOT~cc122288e5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.061 s) : 0, 1061066
Total [baseline] (8.86 s) : 0, 8860310
Agent [candidate] (1.067 s) : 0, 1066977
Total [candidate] (8.848 s) : 0, 8847619
section iast
Agent [baseline] (1.236 s) : 0, 1235669
Total [baseline] (9.577 s) : 0, 9577112
Agent [candidate] (1.228 s) : 0, 1228215
Total [candidate] (9.564 s) : 0, 9564149

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.061 s	-
Agent	iast	1.236 s	174.603 ms (16.5%)
Total	tracing	8.86 s	-
Total	iast	9.577 s	716.803 ms (8.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.067 s	-
Agent	iast	1.228 s	161.238 ms (15.1%)
Total	tracing	8.848 s	-
Total	iast	9.564 s	716.53 ms (8.1%)

gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~a03549e915, baseline=1.61.0-SNAPSHOT~cc122288e5

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.201 ms) : 0, 1201
BytebuddyAgent [baseline] (629.162 ms) : 0, 629162
BytebuddyAgent [candidate] (633.752 ms) : 0, 633752
AgentMeter [baseline] (29.155 ms) : 0, 29155
AgentMeter [candidate] (29.39 ms) : 0, 29390
GlobalTracer [baseline] (257.773 ms) : 0, 257773
GlobalTracer [candidate] (259.402 ms) : 0, 259402
AppSec [baseline] (31.626 ms) : 0, 31626
AppSec [candidate] (31.892 ms) : 0, 31892
Debugger [baseline] (58.857 ms) : 0, 58857
Debugger [candidate] (59.271 ms) : 0, 59271
Remote Config [baseline] (610.596 µs) : 0, 611
Remote Config [candidate] (591.268 µs) : 0, 591
Telemetry [baseline] (8.685 ms) : 0, 8685
Telemetry [candidate] (8.785 ms) : 0, 8785
Flare Poller [baseline] (7.852 ms) : 0, 7852
Flare Poller [candidate] (6.515 ms) : 0, 6515
section iast
crashtracking [baseline] (1.213 ms) : 0, 1213
crashtracking [candidate] (1.186 ms) : 0, 1186
BytebuddyAgent [baseline] (803.215 ms) : 0, 803215
BytebuddyAgent [candidate] (797.59 ms) : 0, 797590
AgentMeter [baseline] (11.439 ms) : 0, 11439
AgentMeter [candidate] (11.329 ms) : 0, 11329
GlobalTracer [baseline] (248.912 ms) : 0, 248912
GlobalTracer [candidate] (247.341 ms) : 0, 247341
AppSec [baseline] (26.503 ms) : 0, 26503
AppSec [candidate] (26.3 ms) : 0, 26300
Debugger [baseline] (62.601 ms) : 0, 62601
Debugger [candidate] (62.941 ms) : 0, 62941
Remote Config [baseline] (523.863 µs) : 0, 524
Remote Config [candidate] (519.426 µs) : 0, 519
Telemetry [baseline] (15.011 ms) : 0, 15011
Telemetry [candidate] (14.921 ms) : 0, 14921
Flare Poller [baseline] (4.777 ms) : 0, 4777
Flare Poller [candidate] (4.888 ms) : 0, 4888
IAST [baseline] (25.277 ms) : 0, 25277
IAST [candidate] (25.101 ms) : 0, 25101

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~a03549e915, baseline=1.61.0-SNAPSHOT~cc122288e5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.061 s) : 0, 1060510
Total [baseline] (11.127 s) : 0, 11126680
Agent [candidate] (1.072 s) : 0, 1071848
Total [candidate] (11.155 s) : 0, 11155383
section appsec
Agent [baseline] (1.254 s) : 0, 1254139
Total [baseline] (11.242 s) : 0, 11242395
Agent [candidate] (1.256 s) : 0, 1255659
Total [candidate] (11.151 s) : 0, 11150516
section iast
Agent [baseline] (1.228 s) : 0, 1228121
Total [baseline] (11.341 s) : 0, 11340507
Agent [candidate] (1.239 s) : 0, 1238949
Total [candidate] (11.426 s) : 0, 11425818
section profiling
Agent [baseline] (1.184 s) : 0, 1184002
Total [baseline] (11.053 s) : 0, 11053079
Agent [candidate] (1.185 s) : 0, 1185050
Total [candidate] (11.017 s) : 0, 11017148

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.061 s	-
Agent	appsec	1.254 s	193.628 ms (18.3%)
Agent	iast	1.228 s	167.611 ms (15.8%)
Agent	profiling	1.184 s	123.492 ms (11.6%)
Total	tracing	11.127 s	-
Total	appsec	11.242 s	115.715 ms (1.0%)
Total	iast	11.341 s	213.827 ms (1.9%)
Total	profiling	11.053 s	-73.601 ms (-0.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.072 s	-
Agent	appsec	1.256 s	183.811 ms (17.1%)
Agent	iast	1.239 s	167.101 ms (15.6%)
Agent	profiling	1.185 s	113.201 ms (10.6%)
Total	tracing	11.155 s	-
Total	appsec	11.151 s	-4.867 ms (-0.0%)
Total	iast	11.426 s	270.435 ms (2.4%)
Total	profiling	11.017 s	-138.235 ms (-1.2%)

gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~a03549e915, baseline=1.61.0-SNAPSHOT~cc122288e5

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.196 ms) : 0, 1196
BytebuddyAgent [baseline] (629.637 ms) : 0, 629637
BytebuddyAgent [candidate] (635.129 ms) : 0, 635129
AgentMeter [baseline] (29.237 ms) : 0, 29237
AgentMeter [candidate] (29.668 ms) : 0, 29668
GlobalTracer [baseline] (258.1 ms) : 0, 258100
GlobalTracer [candidate] (260.617 ms) : 0, 260617
AppSec [baseline] (31.56 ms) : 0, 31560
AppSec [candidate] (32.123 ms) : 0, 32123
Debugger [baseline] (59.615 ms) : 0, 59615
Debugger [candidate] (60.312 ms) : 0, 60312
Remote Config [baseline] (625.443 µs) : 0, 625
Remote Config [candidate] (594.631 µs) : 0, 595
Telemetry [baseline] (8.715 ms) : 0, 8715
Telemetry [candidate] (8.774 ms) : 0, 8774
Flare Poller [baseline] (5.726 ms) : 0, 5726
Flare Poller [candidate] (7.198 ms) : 0, 7198
section appsec
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.199 ms) : 0, 1199
BytebuddyAgent [baseline] (662.571 ms) : 0, 662571
BytebuddyAgent [candidate] (663.923 ms) : 0, 663923
AgentMeter [baseline] (12.066 ms) : 0, 12066
AgentMeter [candidate] (12.156 ms) : 0, 12156
GlobalTracer [baseline] (259.883 ms) : 0, 259883
GlobalTracer [candidate] (260.075 ms) : 0, 260075
IAST [baseline] (24.082 ms) : 0, 24082
IAST [candidate] (24.088 ms) : 0, 24088
AppSec [baseline] (178.467 ms) : 0, 178467
AppSec [candidate] (178.457 ms) : 0, 178457
Debugger [baseline] (66.277 ms) : 0, 66277
Debugger [candidate] (66.111 ms) : 0, 66111
Remote Config [baseline] (572.22 µs) : 0, 572
Remote Config [candidate] (572.778 µs) : 0, 573
Telemetry [baseline] (8.999 ms) : 0, 8999
Telemetry [candidate] (8.945 ms) : 0, 8945
Flare Poller [baseline] (3.644 ms) : 0, 3644
Flare Poller [candidate] (3.628 ms) : 0, 3628
section iast
crashtracking [baseline] (1.198 ms) : 0, 1198
crashtracking [candidate] (1.231 ms) : 0, 1231
BytebuddyAgent [baseline] (796.738 ms) : 0, 796738
BytebuddyAgent [candidate] (804.312 ms) : 0, 804312
AgentMeter [baseline] (11.326 ms) : 0, 11326
AgentMeter [candidate] (11.609 ms) : 0, 11609
GlobalTracer [baseline] (247.52 ms) : 0, 247520
GlobalTracer [candidate] (249.299 ms) : 0, 249299
IAST [baseline] (25.038 ms) : 0, 25038
IAST [candidate] (25.366 ms) : 0, 25366
AppSec [baseline] (26.227 ms) : 0, 26227
AppSec [candidate] (26.617 ms) : 0, 26617
Debugger [baseline] (63.582 ms) : 0, 63582
Debugger [candidate] (63.972 ms) : 0, 63972
Remote Config [baseline] (526.168 µs) : 0, 526
Remote Config [candidate] (522.621 µs) : 0, 523
Telemetry [baseline] (14.971 ms) : 0, 14971
Telemetry [candidate] (14.841 ms) : 0, 14841
Flare Poller [baseline] (4.888 ms) : 0, 4888
Flare Poller [candidate] (4.864 ms) : 0, 4864
section profiling
crashtracking [baseline] (1.189 ms) : 0, 1189
crashtracking [candidate] (1.171 ms) : 0, 1171
BytebuddyAgent [baseline] (683.848 ms) : 0, 683848
BytebuddyAgent [candidate] (684.973 ms) : 0, 684973
AgentMeter [baseline] (8.588 ms) : 0, 8588
AgentMeter [candidate] (8.634 ms) : 0, 8634
GlobalTracer [baseline] (215.787 ms) : 0, 215787
GlobalTracer [candidate] (215.759 ms) : 0, 215759
AppSec [baseline] (31.993 ms) : 0, 31993
AppSec [candidate] (31.943 ms) : 0, 31943
Debugger [baseline] (63.326 ms) : 0, 63326
Debugger [candidate] (62.038 ms) : 0, 62038
Remote Config [baseline] (595.883 µs) : 0, 596
Remote Config [candidate] (593.943 µs) : 0, 594
Telemetry [baseline] (10.462 ms) : 0, 10462
Telemetry [candidate] (11.345 ms) : 0, 11345
Flare Poller [baseline] (3.501 ms) : 0, 3501
Flare Poller [candidate] (3.49 ms) : 0, 3490
ProfilingAgent [baseline] (93.565 ms) : 0, 93565
ProfilingAgent [candidate] (94.083 ms) : 0, 94083
Profiling [baseline] (94.126 ms) : 0, 94126
Profiling [candidate] (94.649 ms) : 0, 94649

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/add-benchmarks
git_commit_date	1773169596	1773171313
git_commit_sha	`cc12228`	`a03549e`
release_version	1.61.0-SNAPSHOT~cc122288e5	1.61.0-SNAPSHOT~a03549e915

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1773173498	1773173498
ci_job_id	1494355678	1494355678
ci_pipeline_id	101726144	101726144
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-dnpmk2p2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-dnpmk2p2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 1 performance regressions! Performance is the same for 18 metrics, 16 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:petclinic:profiling:high_load	worse [+0.793ms; +1.571ms] or [+4.402%; +8.718%]	unsure [+0.554ms; +2.259ms] or [+1.864%; +7.597%]	unstable [-37.846op/s; +8.158op/s] or [-14.961%; +3.225%]	19.198ms	31.136ms	238.125op/s	18.016ms	29.730ms	252.969op/s
scenario:load:petclinic:no_agent:high_load	better [-1.903ms; -0.550ms] or [-10.234%; -2.960%]	unsure [-3.422ms; -0.509ms] or [-10.987%; -1.633%]	unstable [-8.776op/s; +43.026op/s] or [-3.597%; +17.634%]	17.369ms	29.177ms	261.125op/s	18.596ms	31.142ms	244.000op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~a03549e915, baseline=1.61.0-SNAPSHOT~cc122288e5
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.131 ms) : 18935, 19328
.   : milestone, 19131,
appsec (19.39 ms) : 19186, 19593
.   : milestone, 19390,
code_origins (17.985 ms) : 17805, 18164
.   : milestone, 17985,
iast (17.798 ms) : 17619, 17976
.   : milestone, 17798,
profiling (18.447 ms) : 18262, 18632
.   : milestone, 18447,
tracing (17.779 ms) : 17605, 17954
.   : milestone, 17779,
section candidate
no_agent (17.87 ms) : 17691, 18050
.   : milestone, 17870,
appsec (19.523 ms) : 19322, 19723
.   : milestone, 19523,
code_origins (18.071 ms) : 17891, 18251
.   : milestone, 18071,
iast (17.754 ms) : 17577, 17931
.   : milestone, 17754,
profiling (19.605 ms) : 19405, 19804
.   : milestone, 19605,
tracing (17.733 ms) : 17557, 17908
.   : milestone, 17733,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.131 ms [18.935 ms, 19.328 ms]	-
appsec	19.39 ms [19.186 ms, 19.593 ms]	258.212 µs (1.3%)
code_origins	17.985 ms [17.805 ms, 18.164 ms]	-1.147 ms (-6.0%)
iast	17.798 ms [17.619 ms, 17.976 ms]	-1.334 ms (-7.0%)
profiling	18.447 ms [18.262 ms, 18.632 ms]	-684.61 µs (-3.6%)
tracing	17.779 ms [17.605 ms, 17.954 ms]	-1.352 ms (-7.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	17.87 ms [17.691 ms, 18.05 ms]	-
appsec	19.523 ms [19.322 ms, 19.723 ms]	1.652 ms (9.2%)
code_origins	18.071 ms [17.891 ms, 18.251 ms]	200.601 µs (1.1%)
iast	17.754 ms [17.577 ms, 17.931 ms]	-116.551 µs (-0.7%)
profiling	19.605 ms [19.405 ms, 19.804 ms]	1.734 ms (9.7%)
tracing	17.733 ms [17.557 ms, 17.908 ms]	-137.636 µs (-0.8%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~a03549e915, baseline=1.61.0-SNAPSHOT~cc122288e5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.184 ms) : 1172, 1196
.   : milestone, 1184,
iast (3.117 ms) : 3076, 3158
.   : milestone, 3117,
iast_FULL (5.869 ms) : 5809, 5928
.   : milestone, 5869,
iast_GLOBAL (3.525 ms) : 3473, 3578
.   : milestone, 3525,
profiling (2.105 ms) : 2086, 2125
.   : milestone, 2105,
tracing (1.792 ms) : 1777, 1807
.   : milestone, 1792,
section candidate
no_agent (1.196 ms) : 1184, 1207
.   : milestone, 1196,
iast (3.112 ms) : 3069, 3154
.   : milestone, 3112,
iast_FULL (5.869 ms) : 5810, 5927
.   : milestone, 5869,
iast_GLOBAL (3.606 ms) : 3545, 3667
.   : milestone, 3606,
profiling (2.006 ms) : 1988, 2024
.   : milestone, 2006,
tracing (1.752 ms) : 1737, 1766
.   : milestone, 1752,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.184 ms [1.172 ms, 1.196 ms]	-
iast	3.117 ms [3.076 ms, 3.158 ms]	1.933 ms (163.3%)
iast_FULL	5.869 ms [5.809 ms, 5.928 ms]	4.685 ms (395.7%)
iast_GLOBAL	3.525 ms [3.473 ms, 3.578 ms]	2.341 ms (197.8%)
profiling	2.105 ms [2.086 ms, 2.125 ms]	921.465 µs (77.8%)
tracing	1.792 ms [1.777 ms, 1.807 ms]	608.028 µs (51.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.196 ms [1.184 ms, 1.207 ms]	-
iast	3.112 ms [3.069 ms, 3.154 ms]	1.916 ms (160.3%)
iast_FULL	5.869 ms [5.81 ms, 5.927 ms]	4.673 ms (390.8%)
iast_GLOBAL	3.606 ms [3.545 ms, 3.667 ms]	2.41 ms (201.6%)
profiling	2.006 ms [1.988 ms, 2.024 ms]	810.047 µs (67.8%)
tracing	1.752 ms [1.737 ms, 1.766 ms]	555.946 µs (46.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/add-benchmarks
git_commit_date	1773169596	1773171313
git_commit_sha	`cc12228`	`a03549e`
release_version	1.61.0-SNAPSHOT~cc122288e5	1.61.0-SNAPSHOT~a03549e915

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1773173275	1773173275
ci_job_id	1494355680	1494355680
ci_pipeline_id	101726144	101726144
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-b6hi2wa0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-b6hi2wa0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~a03549e915, baseline=1.61.0-SNAPSHOT~cc122288e5
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.598 s) : 15598000, 15598000
.   : milestone, 15598000,
appsec (15.012 s) : 15012000, 15012000
.   : milestone, 15012000,
iast (18.249 s) : 18249000, 18249000
.   : milestone, 18249000,
iast_GLOBAL (17.618 s) : 17618000, 17618000
.   : milestone, 17618000,
profiling (15.03 s) : 15030000, 15030000
.   : milestone, 15030000,
tracing (15.301 s) : 15301000, 15301000
.   : milestone, 15301000,
section candidate
no_agent (14.986 s) : 14986000, 14986000
.   : milestone, 14986000,
appsec (14.947 s) : 14947000, 14947000
.   : milestone, 14947000,
iast (17.862 s) : 17862000, 17862000
.   : milestone, 17862000,
iast_GLOBAL (18.348 s) : 18348000, 18348000
.   : milestone, 18348000,
profiling (14.567 s) : 14567000, 14567000
.   : milestone, 14567000,
tracing (14.999 s) : 14999000, 14999000
.   : milestone, 14999000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.598 s [15.598 s, 15.598 s]	-
appsec	15.012 s [15.012 s, 15.012 s]	-586.0 ms (-3.8%)
iast	18.249 s [18.249 s, 18.249 s]	2.651 s (17.0%)
iast_GLOBAL	17.618 s [17.618 s, 17.618 s]	2.02 s (13.0%)
profiling	15.03 s [15.03 s, 15.03 s]	-568.0 ms (-3.6%)
tracing	15.301 s [15.301 s, 15.301 s]	-297.0 ms (-1.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.986 s [14.986 s, 14.986 s]	-
appsec	14.947 s [14.947 s, 14.947 s]	-39.0 ms (-0.3%)
iast	17.862 s [17.862 s, 17.862 s]	2.876 s (19.2%)
iast_GLOBAL	18.348 s [18.348 s, 18.348 s]	3.362 s (22.4%)
profiling	14.567 s [14.567 s, 14.567 s]	-419.0 ms (-2.8%)
tracing	14.999 s [14.999 s, 14.999 s]	13.0 ms (0.1%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~a03549e915, baseline=1.61.0-SNAPSHOT~cc122288e5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (2.538 ms) : 2483, 2593
.   : milestone, 2538,
iast (2.269 ms) : 2199, 2338
.   : milestone, 2269,
iast_GLOBAL (2.311 ms) : 2241, 2380
.   : milestone, 2311,
profiling (2.101 ms) : 2045, 2156
.   : milestone, 2101,
tracing (2.08 ms) : 2026, 2133
.   : milestone, 2080,
section candidate
no_agent (1.485 ms) : 1473, 1497
.   : milestone, 1485,
appsec (3.803 ms) : 3582, 4024
.   : milestone, 3803,
iast (2.267 ms) : 2198, 2336
.   : milestone, 2267,
iast_GLOBAL (2.316 ms) : 2246, 2386
.   : milestone, 2316,
profiling (2.512 ms) : 2349, 2675
.   : milestone, 2512,
tracing (2.075 ms) : 2021, 2129
.   : milestone, 2075,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.48 ms [1.469 ms, 1.492 ms]	-
appsec	2.538 ms [2.483 ms, 2.593 ms]	1.057 ms (71.4%)
iast	2.269 ms [2.199 ms, 2.338 ms]	788.105 µs (53.2%)
iast_GLOBAL	2.311 ms [2.241 ms, 2.38 ms]	830.355 µs (56.1%)
profiling	2.101 ms [2.045 ms, 2.156 ms]	620.204 µs (41.9%)
tracing	2.08 ms [2.026 ms, 2.133 ms]	599.268 µs (40.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.485 ms [1.473 ms, 1.497 ms]	-
appsec	3.803 ms [3.582 ms, 4.024 ms]	2.318 ms (156.1%)
iast	2.267 ms [2.198 ms, 2.336 ms]	781.797 µs (52.6%)
iast_GLOBAL	2.316 ms [2.246 ms, 2.386 ms]	831.249 µs (56.0%)
profiling	2.512 ms [2.349 ms, 2.675 ms]	1.027 ms (69.1%)
tracing	2.075 ms [2.021 ms, 2.129 ms]	590.154 µs (39.7%)

sarahchen6 added 2 commits March 9, 2026 16:09

Add apm-sdks-benchmarks to CI

3546e21

Ensure artifacts are published before benchmarks run

09706c5

sarahchen6 force-pushed the sarahchen6/add-benchmarks branch from 930b901 to 09706c5 Compare March 9, 2026 20:09

Merge branch 'master' into sarahchen6/add-benchmarks

a03549e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add apm-sdks-benchmarks to CI#10775

Add apm-sdks-benchmarks to CI#10775
sarahchen6 wants to merge 3 commits intomasterfrom
sarahchen6/add-benchmarks

sarahchen6 commented Mar 9, 2026

Uh oh!

pr-commenter bot commented Mar 9, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

sarahchen6 commented Mar 9, 2026

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

pr-commenter bot commented Mar 9, 2026 •

edited

Loading