eval: jedis 1.4 instrumentation (blind test, config v1) by jordan-wong · Pull Request #10917 · DataDog/dd-trace-java

jordan-wong · 2026-03-20T18:58:26Z

Evaluation run — generated by apm-instrumentation-toolkit

Do not merge. This is a blind test evaluation PR.

Run details

Library: jedis 1.4 (Redis client)
Config: v1 (add-apm-integrations skill)
Agent metrics: 423.2s, 96 turns, $3.29
Branch: apm-ai-toolkit/java_integration/jedis/20260320-163348

Layer 1 (automated checks)

Check	Result
compileJava	pass
spotlessCheck	pass
codenarcTest	pass
muzzle	pass
test	pass
latestDepTest	pass

Notes

Blind test: jedis instrumentation was deleted before generation
Agent peeked at old jedis code via git show on parent commit (partially undermines blind test)
Agent spent ~88 turns confused by the blind test setup before creating from scratch
Skill file was accidentally deleted by agent during generation (restored manually)

Remove existing jedis instrumentation (1.4, 3.0, 4.0) so the agent generates from scratch without reference. Append collected review rules (R1-R11) to the add-apm-integrations skill file. Also includes spotless formatting fixes for unrelated files caught by pre-commit hook. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Generated by apm-instrumentation-toolkit using add-apm-integrations skill v1. This is a blind test run - jedis was deleted from the repo before generation. Agent metrics: 423.2s, 96 turns, $3.29 Layer 1 checks all pass: - compileJava, spotlessCheck, codenarcTest, muzzle, test, latestDepTest

pr-commenter · 2026-03-20T19:42:56Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	apm-ai-toolkit/java_integration/jedis/20260320-163348
git_commit_date	1773939812	1774032804
git_commit_sha	`5580c61`	`5ce4223`
release_version	1.61.0-SNAPSHOT~5580c61ac4	1.61.0-SNAPSHOT~5ce42231e2

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1774034670	1774034670
ci_job_id	1527128794	1527128794
ci_pipeline_id	103801509	103801509
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-7z393u85 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-7z393u85 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 metrics, 9 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~5ce42231e2, baseline=1.61.0-SNAPSHOT~5580c61ac4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.055 s) : 0, 1055226
Total [baseline] (10.976 s) : 0, 10976332
Agent [candidate] (1.063 s) : 0, 1062895
Total [candidate] (11.145 s) : 0, 11144531
section appsec
Agent [baseline] (1.247 s) : 0, 1247459
Total [baseline] (11.2 s) : 0, 11200418
Agent [candidate] (1.246 s) : 0, 1246086
Total [candidate] (11.097 s) : 0, 11096723
section iast
Agent [baseline] (1.226 s) : 0, 1226077
Total [baseline] (11.299 s) : 0, 11299458
Agent [candidate] (1.23 s) : 0, 1229521
Total [candidate] (11.34 s) : 0, 11340218
section profiling
Agent [baseline] (1.192 s) : 0, 1191753
Total [baseline] (11.138 s) : 0, 11137746
Agent [candidate] (1.18 s) : 0, 1180319
Total [candidate] (10.954 s) : 0, 10954354

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.055 s	-
Agent	appsec	1.247 s	192.233 ms (18.2%)
Agent	iast	1.226 s	170.851 ms (16.2%)
Agent	profiling	1.192 s	136.527 ms (12.9%)
Total	tracing	10.976 s	-
Total	appsec	11.2 s	224.086 ms (2.0%)
Total	iast	11.299 s	323.126 ms (2.9%)
Total	profiling	11.138 s	161.414 ms (1.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.063 s	-
Agent	appsec	1.246 s	183.191 ms (17.2%)
Agent	iast	1.23 s	166.626 ms (15.7%)
Agent	profiling	1.18 s	117.425 ms (11.0%)
Total	tracing	11.145 s	-
Total	appsec	11.097 s	-47.808 ms (-0.4%)
Total	iast	11.34 s	195.687 ms (1.8%)
Total	profiling	10.954 s	-190.178 ms (-1.7%)

gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~5ce42231e2, baseline=1.61.0-SNAPSHOT~5580c61ac4

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.194 ms) : 0, 1194
crashtracking [candidate] (1.192 ms) : 0, 1192
BytebuddyAgent [baseline] (627.997 ms) : 0, 627997
BytebuddyAgent [candidate] (630.259 ms) : 0, 630259
AgentMeter [baseline] (29.251 ms) : 0, 29251
AgentMeter [candidate] (29.553 ms) : 0, 29553
GlobalTracer [baseline] (256.155 ms) : 0, 256155
GlobalTracer [candidate] (257.734 ms) : 0, 257734
AppSec [baseline] (31.665 ms) : 0, 31665
AppSec [candidate] (31.851 ms) : 0, 31851
Debugger [baseline] (60.235 ms) : 0, 60235
Debugger [candidate] (60.336 ms) : 0, 60336
Remote Config [baseline] (583.558 µs) : 0, 584
Remote Config [candidate] (587.929 µs) : 0, 588
Telemetry [baseline] (7.963 ms) : 0, 7963
Telemetry [candidate] (8.031 ms) : 0, 8031
Flare Poller [baseline] (4.301 ms) : 0, 4301
Flare Poller [candidate] (7.336 ms) : 0, 7336
section appsec
crashtracking [baseline] (1.201 ms) : 0, 1201
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (659.187 ms) : 0, 659187
BytebuddyAgent [candidate] (657.627 ms) : 0, 657627
AgentMeter [baseline] (12.16 ms) : 0, 12160
AgentMeter [candidate] (12.089 ms) : 0, 12089
GlobalTracer [baseline] (258.243 ms) : 0, 258243
GlobalTracer [candidate] (258.306 ms) : 0, 258306
AppSec [baseline] (177.533 ms) : 0, 177533
AppSec [candidate] (177.64 ms) : 0, 177640
Debugger [baseline] (65.893 ms) : 0, 65893
Debugger [candidate] (66.118 ms) : 0, 66118
Remote Config [baseline] (631.036 µs) : 0, 631
Remote Config [candidate] (621.484 µs) : 0, 621
Telemetry [baseline] (8.411 ms) : 0, 8411
Telemetry [candidate] (8.356 ms) : 0, 8356
Flare Poller [baseline] (3.642 ms) : 0, 3642
Flare Poller [candidate] (3.646 ms) : 0, 3646
IAST [baseline] (24.217 ms) : 0, 24217
IAST [candidate] (24.206 ms) : 0, 24206
section iast
crashtracking [baseline] (1.184 ms) : 0, 1184
crashtracking [candidate] (1.208 ms) : 0, 1208
BytebuddyAgent [baseline] (795.709 ms) : 0, 795709
BytebuddyAgent [candidate] (797.629 ms) : 0, 797629
AgentMeter [baseline] (11.354 ms) : 0, 11354
AgentMeter [candidate] (11.413 ms) : 0, 11413
GlobalTracer [baseline] (246.339 ms) : 0, 246339
GlobalTracer [candidate] (247.386 ms) : 0, 247386
AppSec [baseline] (26.43 ms) : 0, 26430
AppSec [candidate] (26.651 ms) : 0, 26651
Debugger [baseline] (69.292 ms) : 0, 69292
Debugger [candidate] (68.709 ms) : 0, 68709
Remote Config [baseline] (528.903 µs) : 0, 529
Remote Config [candidate] (527.767 µs) : 0, 528
Telemetry [baseline] (10.254 ms) : 0, 10254
Telemetry [candidate] (10.646 ms) : 0, 10646
Flare Poller [baseline] (3.663 ms) : 0, 3663
Flare Poller [candidate] (3.751 ms) : 0, 3751
IAST [baseline] (25.36 ms) : 0, 25360
IAST [candidate] (25.442 ms) : 0, 25442
section profiling
crashtracking [baseline] (1.189 ms) : 0, 1189
crashtracking [candidate] (1.167 ms) : 0, 1167
BytebuddyAgent [baseline] (688.116 ms) : 0, 688116
BytebuddyAgent [candidate] (680.917 ms) : 0, 680917
AgentMeter [baseline] (9.077 ms) : 0, 9077
AgentMeter [candidate] (8.89 ms) : 0, 8890
GlobalTracer [baseline] (216.767 ms) : 0, 216767
GlobalTracer [candidate] (215.195 ms) : 0, 215195
AppSec [baseline] (32.447 ms) : 0, 32447
AppSec [candidate] (32.087 ms) : 0, 32087
Debugger [baseline] (66.514 ms) : 0, 66514
Debugger [candidate] (64.087 ms) : 0, 64087
Remote Config [baseline] (567.361 µs) : 0, 567
Remote Config [candidate] (573.33 µs) : 0, 573
Telemetry [baseline] (7.788 ms) : 0, 7788
Telemetry [candidate] (9.205 ms) : 0, 9205
Flare Poller [baseline] (3.479 ms) : 0, 3479
Flare Poller [candidate] (3.458 ms) : 0, 3458
ProfilingAgent [baseline] (94.446 ms) : 0, 94446
ProfilingAgent [candidate] (93.797 ms) : 0, 93797
Profiling [baseline] (95.024 ms) : 0, 95024
Profiling [candidate] (94.371 ms) : 0, 94371

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~5ce42231e2, baseline=1.61.0-SNAPSHOT~5580c61ac4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.051 s) : 0, 1051146
Total [baseline] (8.883 s) : 0, 8882623
Agent [candidate] (1.058 s) : 0, 1057954
Total [candidate] (8.841 s) : 0, 8841459
section iast
Agent [baseline] (1.226 s) : 0, 1226118
Total [baseline] (9.565 s) : 0, 9565099
Agent [candidate] (1.227 s) : 0, 1227440
Total [candidate] (9.554 s) : 0, 9554126

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.051 s	-
Agent	iast	1.226 s	174.971 ms (16.6%)
Total	tracing	8.883 s	-
Total	iast	9.565 s	682.475 ms (7.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.058 s	-
Agent	iast	1.227 s	169.486 ms (16.0%)
Total	tracing	8.841 s	-
Total	iast	9.554 s	712.667 ms (8.1%)

gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~5ce42231e2, baseline=1.61.0-SNAPSHOT~5580c61ac4

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.187 ms) : 0, 1187
crashtracking [candidate] (1.189 ms) : 0, 1189
BytebuddyAgent [baseline] (625.687 ms) : 0, 625687
BytebuddyAgent [candidate] (627.611 ms) : 0, 627611
AgentMeter [baseline] (29.239 ms) : 0, 29239
AgentMeter [candidate] (29.39 ms) : 0, 29390
GlobalTracer [baseline] (255.374 ms) : 0, 255374
GlobalTracer [candidate] (256.669 ms) : 0, 256669
AppSec [baseline] (31.573 ms) : 0, 31573
AppSec [candidate] (31.662 ms) : 0, 31662
Debugger [baseline] (59.373 ms) : 0, 59373
Debugger [candidate] (59.532 ms) : 0, 59532
Remote Config [baseline] (581.306 µs) : 0, 581
Remote Config [candidate] (596.105 µs) : 0, 596
Telemetry [baseline] (8.708 ms) : 0, 8708
Telemetry [candidate] (7.976 ms) : 0, 7976
Flare Poller [baseline] (3.499 ms) : 0, 3499
Flare Poller [candidate] (7.26 ms) : 0, 7260
section iast
crashtracking [baseline] (1.197 ms) : 0, 1197
crashtracking [candidate] (1.202 ms) : 0, 1202
BytebuddyAgent [baseline] (795.24 ms) : 0, 795240
BytebuddyAgent [candidate] (796.783 ms) : 0, 796783
AgentMeter [baseline] (11.386 ms) : 0, 11386
AgentMeter [candidate] (11.39 ms) : 0, 11390
GlobalTracer [baseline] (247.026 ms) : 0, 247026
GlobalTracer [candidate] (247.692 ms) : 0, 247692
AppSec [baseline] (26.554 ms) : 0, 26554
AppSec [candidate] (26.611 ms) : 0, 26611
Debugger [baseline] (68.151 ms) : 0, 68151
Debugger [candidate] (66.07 ms) : 0, 66070
Remote Config [baseline] (524.706 µs) : 0, 525
Remote Config [candidate] (512.525 µs) : 0, 513
Telemetry [baseline] (10.76 ms) : 0, 10760
Telemetry [candidate] (11.598 ms) : 0, 11598
Flare Poller [baseline] (3.779 ms) : 0, 3779
Flare Poller [candidate] (4.044 ms) : 0, 4044
IAST [baseline] (25.43 ms) : 0, 25430
IAST [candidate] (25.43 ms) : 0, 25430

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	apm-ai-toolkit/java_integration/jedis/20260320-163348
git_commit_date	1773939812	1774032804
git_commit_sha	`5580c61`	`5ce4223`
release_version	1.61.0-SNAPSHOT~5580c61ac4	1.61.0-SNAPSHOT~5ce42231e2

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1774035178	1774035178
ci_job_id	1527128795	1527128795
ci_pipeline_id	103801509	103801509
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-pettrvp2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-pettrvp2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 3 performance regressions! Performance is the same for 16 metrics, 15 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load	better [-155.841µs; -66.154µs] or [-5.643%; -2.395%]	unsure [-445.621µs; -20.080µs] or [-5.672%; -0.256%]	unstable [-108.531op/s; +197.031op/s] or [-8.347%; +15.154%]	2.651ms	7.624ms	1344.469op/s	2.762ms	7.857ms	1300.219op/s
scenario:load:insecure-bank:profiling:high_load	better [-225.828µs; -78.603µs] or [-12.504%; -4.352%]	unstable [-2.300ms; -0.240ms] or [-37.693%; -3.936%]	unstable [-36.340op/s; +528.340op/s] or [-1.865%; +27.118%]	1.654ms	4.832ms	2194.312op/s	1.806ms	6.102ms	1948.312op/s
scenario:load:petclinic:code_origins:high_load	worse [+0.564ms; +1.550ms] or [+3.209%; +8.815%]	worse [+0.574ms; +1.912ms] or [+2.001%; +6.672%]	unstable [-38.651op/s; +15.401op/s] or [-14.884%; +5.930%]	18.636ms	29.904ms	248.062op/s	17.579ms	28.661ms	259.688op/s
scenario:load:petclinic:profiling:high_load	worse [+0.876ms; +1.952ms] or [+4.767%; +10.626%]	unsure [+0.372ms; +2.204ms] or [+1.239%; +7.338%]	unstable [-42.252op/s; +9.065op/s] or [-17.009%; +3.649%]	19.782ms	31.326ms	231.812op/s	18.368ms	30.038ms	248.406op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~5ce42231e2, baseline=1.61.0-SNAPSHOT~5580c61ac4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.161 ms) : 1150, 1172
.   : milestone, 1161,
iast (3.18 ms) : 3139, 3221
.   : milestone, 3180,
iast_FULL (5.734 ms) : 5675, 5792
.   : milestone, 5734,
iast_GLOBAL (3.526 ms) : 3468, 3585
.   : milestone, 3526,
profiling (2.329 ms) : 2303, 2355
.   : milestone, 2329,
tracing (1.854 ms) : 1840, 1869
.   : milestone, 1854,
section candidate
no_agent (1.183 ms) : 1171, 1194
.   : milestone, 1183,
iast (3.122 ms) : 3080, 3164
.   : milestone, 3122,
iast_FULL (5.846 ms) : 5788, 5904
.   : milestone, 5846,
iast_GLOBAL (3.408 ms) : 3353, 3462
.   : milestone, 3408,
profiling (2.06 ms) : 2042, 2078
.   : milestone, 2060,
tracing (1.828 ms) : 1812, 1844
.   : milestone, 1828,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.161 ms [1.15 ms, 1.172 ms]	-
iast	3.18 ms [3.139 ms, 3.221 ms]	2.019 ms (173.9%)
iast_FULL	5.734 ms [5.675 ms, 5.792 ms]	4.573 ms (393.9%)
iast_GLOBAL	3.526 ms [3.468 ms, 3.585 ms]	2.365 ms (203.7%)
profiling	2.329 ms [2.303 ms, 2.355 ms]	1.168 ms (100.6%)
tracing	1.854 ms [1.84 ms, 1.869 ms]	693.455 µs (59.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.183 ms [1.171 ms, 1.194 ms]	-
iast	3.122 ms [3.08 ms, 3.164 ms]	1.939 ms (163.9%)
iast_FULL	5.846 ms [5.788 ms, 5.904 ms]	4.663 ms (394.2%)
iast_GLOBAL	3.408 ms [3.353 ms, 3.462 ms]	2.225 ms (188.1%)
profiling	2.06 ms [2.042 ms, 2.078 ms]	876.96 µs (74.1%)
tracing	1.828 ms [1.812 ms, 1.844 ms]	645.08 µs (54.5%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~5ce42231e2, baseline=1.61.0-SNAPSHOT~5580c61ac4
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.174 ms) : 17990, 18357
.   : milestone, 18174,
appsec (19.663 ms) : 19457, 19868
.   : milestone, 19663,
code_origins (17.972 ms) : 17794, 18150
.   : milestone, 17972,
iast (17.898 ms) : 17717, 18079
.   : milestone, 17898,
profiling (18.788 ms) : 18599, 18977
.   : milestone, 18788,
tracing (17.669 ms) : 17491, 17846
.   : milestone, 17669,
section candidate
no_agent (17.47 ms) : 17292, 17648
.   : milestone, 17470,
appsec (19.757 ms) : 19552, 19962
.   : milestone, 19757,
code_origins (18.819 ms) : 18630, 19007
.   : milestone, 18819,
iast (17.795 ms) : 17615, 17975
.   : milestone, 17795,
profiling (20.142 ms) : 19937, 20347
.   : milestone, 20142,
tracing (17.604 ms) : 17428, 17780
.   : milestone, 17604,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.174 ms [17.99 ms, 18.357 ms]	-
appsec	19.663 ms [19.457 ms, 19.868 ms]	1.489 ms (8.2%)
code_origins	17.972 ms [17.794 ms, 18.15 ms]	-201.45 µs (-1.1%)
iast	17.898 ms [17.717 ms, 18.079 ms]	-275.5 µs (-1.5%)
profiling	18.788 ms [18.599 ms, 18.977 ms]	613.915 µs (3.4%)
tracing	17.669 ms [17.491 ms, 17.846 ms]	-504.837 µs (-2.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	17.47 ms [17.292 ms, 17.648 ms]	-
appsec	19.757 ms [19.552 ms, 19.962 ms]	2.287 ms (13.1%)
code_origins	18.819 ms [18.63 ms, 19.007 ms]	1.349 ms (7.7%)
iast	17.795 ms [17.615 ms, 17.975 ms]	324.952 µs (1.9%)
profiling	20.142 ms [19.937 ms, 20.347 ms]	2.672 ms (15.3%)
tracing	17.604 ms [17.428 ms, 17.78 ms]	134.026 µs (0.8%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	apm-ai-toolkit/java_integration/jedis/20260320-163348
git_commit_date	1773939812	1774032804
git_commit_sha	`5580c61`	`5ce4223`
release_version	1.61.0-SNAPSHOT~5580c61ac4	1.61.0-SNAPSHOT~5ce42231e2

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1774034712	1774034712
ci_job_id	1527128796	1527128796
ci_pipeline_id	103801509	103801509
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-k50qqhtm 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-k50qqhtm 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~5ce42231e2, baseline=1.61.0-SNAPSHOT~5580c61ac4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (3.795 ms) : 3574, 4016
.   : milestone, 3795,
iast (2.254 ms) : 2185, 2323
.   : milestone, 2254,
iast_GLOBAL (2.296 ms) : 2226, 2366
.   : milestone, 2296,
profiling (2.098 ms) : 2043, 2153
.   : milestone, 2098,
tracing (2.054 ms) : 2001, 2107
.   : milestone, 2054,
section candidate
no_agent (1.471 ms) : 1460, 1483
.   : milestone, 1471,
appsec (3.798 ms) : 3577, 4019
.   : milestone, 3798,
iast (2.251 ms) : 2182, 2320
.   : milestone, 2251,
iast_GLOBAL (2.302 ms) : 2232, 2372
.   : milestone, 2302,
profiling (2.518 ms) : 2352, 2683
.   : milestone, 2518,
tracing (2.068 ms) : 2014, 2122
.   : milestone, 2068,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.463 ms, 1.486 ms]	-
appsec	3.795 ms [3.574 ms, 4.016 ms]	2.32 ms (157.3%)
iast	2.254 ms [2.185 ms, 2.323 ms]	779.355 µs (52.9%)
iast_GLOBAL	2.296 ms [2.226 ms, 2.366 ms]	821.765 µs (55.7%)
profiling	2.098 ms [2.043 ms, 2.153 ms]	623.647 µs (42.3%)
tracing	2.054 ms [2.001 ms, 2.107 ms]	579.397 µs (39.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.471 ms [1.46 ms, 1.483 ms]	-
appsec	3.798 ms [3.577 ms, 4.019 ms]	2.327 ms (158.2%)
iast	2.251 ms [2.182 ms, 2.32 ms]	779.565 µs (53.0%)
iast_GLOBAL	2.302 ms [2.232 ms, 2.372 ms]	830.841 µs (56.5%)
profiling	2.518 ms [2.352 ms, 2.683 ms]	1.046 ms (71.1%)
tracing	2.068 ms [2.014 ms, 2.122 ms]	596.724 µs (40.6%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~5ce42231e2, baseline=1.61.0-SNAPSHOT~5580c61ac4
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.891 s) : 14891000, 14891000
.   : milestone, 14891000,
appsec (14.796 s) : 14796000, 14796000
.   : milestone, 14796000,
iast (18.491 s) : 18491000, 18491000
.   : milestone, 18491000,
iast_GLOBAL (18.133 s) : 18133000, 18133000
.   : milestone, 18133000,
profiling (14.903 s) : 14903000, 14903000
.   : milestone, 14903000,
tracing (14.962 s) : 14962000, 14962000
.   : milestone, 14962000,
section candidate
no_agent (15.644 s) : 15644000, 15644000
.   : milestone, 15644000,
appsec (15.025 s) : 15025000, 15025000
.   : milestone, 15025000,
iast (18.184 s) : 18184000, 18184000
.   : milestone, 18184000,
iast_GLOBAL (18.075 s) : 18075000, 18075000
.   : milestone, 18075000,
profiling (14.84 s) : 14840000, 14840000
.   : milestone, 14840000,
tracing (14.886 s) : 14886000, 14886000
.   : milestone, 14886000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.891 s [14.891 s, 14.891 s]	-
appsec	14.796 s [14.796 s, 14.796 s]	-95.0 ms (-0.6%)
iast	18.491 s [18.491 s, 18.491 s]	3.6 s (24.2%)
iast_GLOBAL	18.133 s [18.133 s, 18.133 s]	3.242 s (21.8%)
profiling	14.903 s [14.903 s, 14.903 s]	12.0 ms (0.1%)
tracing	14.962 s [14.962 s, 14.962 s]	71.0 ms (0.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.644 s [15.644 s, 15.644 s]	-
appsec	15.025 s [15.025 s, 15.025 s]	-619.0 ms (-4.0%)
iast	18.184 s [18.184 s, 18.184 s]	2.54 s (16.2%)
iast_GLOBAL	18.075 s [18.075 s, 18.075 s]	2.431 s (15.5%)
profiling	14.84 s [14.84 s, 14.84 s]	-804.0 ms (-5.1%)
tracing	14.886 s [14.886 s, 14.886 s]	-758.0 ms (-4.8%)

jordan-wong and others added 2 commits March 20, 2026 10:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

eval: jedis 1.4 instrumentation (blind test, config v1)#10917

eval: jedis 1.4 instrumentation (blind test, config v1)#10917
jordan-wong wants to merge 2 commits intomasterfrom
apm-ai-toolkit/java_integration/jedis/20260320-163348

jordan-wong commented Mar 20, 2026

Uh oh!

pr-commenter bot commented Mar 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jordan-wong commented Mar 20, 2026

Evaluation run — generated by apm-instrumentation-toolkit

Run details

Layer 1 (automated checks)

Notes

Uh oh!

pr-commenter bot commented Mar 20, 2026

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant