-
Notifications
You must be signed in to change notification settings - Fork 324
Exclude 'testWallclockProfilingWithoutTracing' from running on Oracle JDK 8 #10492
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Added custom pipeline to make sure we are running or oracle8 before we pronounce this fixed. https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-java/-/pipelines/94032085 |
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 63 metrics, 8 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.59.0-SNAPSHOT~46230e2bb1, baseline=1.59.0-SNAPSHOT~f2f109ef18
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1067297
Total [baseline] (8.752 s) : 0, 8752028
Agent [candidate] (1.066 s) : 0, 1066213
Total [candidate] (8.702 s) : 0, 8702103
section iast
Agent [baseline] (1.227 s) : 0, 1227383
Total [baseline] (9.322 s) : 0, 9322250
Agent [candidate] (1.233 s) : 0, 1233215
Total [candidate] (9.366 s) : 0, 9365893
gantt
title insecure-bank - break down per module: candidate=1.59.0-SNAPSHOT~46230e2bb1, baseline=1.59.0-SNAPSHOT~f2f109ef18
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.208 ms) : 0, 1208
crashtracking [candidate] (1.189 ms) : 0, 1189
BytebuddyAgent [baseline] (631.767 ms) : 0, 631767
BytebuddyAgent [candidate] (629.389 ms) : 0, 629389
AgentMeter [baseline] (28.845 ms) : 0, 28845
AgentMeter [candidate] (28.748 ms) : 0, 28748
GlobalTracer [baseline] (257.7 ms) : 0, 257700
GlobalTracer [candidate] (257.826 ms) : 0, 257826
AppSec [baseline] (32.904 ms) : 0, 32904
AppSec [candidate] (32.75 ms) : 0, 32750
Debugger [baseline] (62.015 ms) : 0, 62015
Debugger [candidate] (62.112 ms) : 0, 62112
Remote Config [baseline] (618.701 µs) : 0, 619
Remote Config [candidate] (601.038 µs) : 0, 601
Telemetry [baseline] (11.442 ms) : 0, 11442
Telemetry [candidate] (12.979 ms) : 0, 12979
Flare Poller [baseline] (5.384 ms) : 0, 5384
Flare Poller [candidate] (5.332 ms) : 0, 5332
section iast
crashtracking [baseline] (1.187 ms) : 0, 1187
crashtracking [candidate] (1.191 ms) : 0, 1191
BytebuddyAgent [baseline] (793.484 ms) : 0, 793484
BytebuddyAgent [candidate] (798.053 ms) : 0, 798053
AgentMeter [baseline] (11.099 ms) : 0, 11099
AgentMeter [candidate] (11.154 ms) : 0, 11154
GlobalTracer [baseline] (247.185 ms) : 0, 247185
GlobalTracer [candidate] (248.17 ms) : 0, 248170
IAST [baseline] (26.952 ms) : 0, 26952
IAST [candidate] (27.059 ms) : 0, 27059
AppSec [baseline] (35.625 ms) : 0, 35625
AppSec [candidate] (33.013 ms) : 0, 33013
Debugger [baseline] (63.919 ms) : 0, 63919
Debugger [candidate] (66.647 ms) : 0, 66647
Remote Config [baseline] (549.811 µs) : 0, 550
Remote Config [candidate] (538.228 µs) : 0, 538
Telemetry [baseline] (8.761 ms) : 0, 8761
Telemetry [candidate] (8.686 ms) : 0, 8686
Flare Poller [baseline] (3.44 ms) : 0, 3440
Flare Poller [candidate] (3.485 ms) : 0, 3485
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.59.0-SNAPSHOT~46230e2bb1, baseline=1.59.0-SNAPSHOT~f2f109ef18
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1062561
Total [baseline] (10.829 s) : 0, 10828705
Agent [candidate] (1.064 s) : 0, 1064401
Total [candidate] (10.906 s) : 0, 10905795
section appsec
Agent [baseline] (1.234 s) : 0, 1234092
Total [baseline] (10.927 s) : 0, 10926776
Agent [candidate] (1.235 s) : 0, 1235362
Total [candidate] (10.899 s) : 0, 10899350
section iast
Agent [baseline] (1.233 s) : 0, 1233270
Total [baseline] (11.196 s) : 0, 11195854
Agent [candidate] (1.23 s) : 0, 1229770
Total [candidate] (11.195 s) : 0, 11195484
section profiling
Agent [baseline] (1.19 s) : 0, 1189990
Total [baseline] (10.899 s) : 0, 10898985
Agent [candidate] (1.186 s) : 0, 1186063
Total [candidate] (10.877 s) : 0, 10876694
gantt
title petclinic - break down per module: candidate=1.59.0-SNAPSHOT~46230e2bb1, baseline=1.59.0-SNAPSHOT~f2f109ef18
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.184 ms) : 0, 1184
crashtracking [candidate] (1.187 ms) : 0, 1187
BytebuddyAgent [baseline] (627.581 ms) : 0, 627581
BytebuddyAgent [candidate] (627.833 ms) : 0, 627833
AgentMeter [baseline] (28.73 ms) : 0, 28730
AgentMeter [candidate] (28.674 ms) : 0, 28674
GlobalTracer [baseline] (257.579 ms) : 0, 257579
GlobalTracer [candidate] (257.513 ms) : 0, 257513
AppSec [baseline] (32.89 ms) : 0, 32890
AppSec [candidate] (32.815 ms) : 0, 32815
Debugger [baseline] (63.378 ms) : 0, 63378
Debugger [candidate] (62.198 ms) : 0, 62198
Remote Config [baseline] (614.718 µs) : 0, 615
Remote Config [candidate] (639.274 µs) : 0, 639
Telemetry [baseline] (9.931 ms) : 0, 9931
Telemetry [candidate] (12.9 ms) : 0, 12900
Flare Poller [baseline] (5.351 ms) : 0, 5351
Flare Poller [candidate] (5.355 ms) : 0, 5355
section appsec
crashtracking [baseline] (1.197 ms) : 0, 1197
crashtracking [candidate] (1.178 ms) : 0, 1178
BytebuddyAgent [baseline] (655.646 ms) : 0, 655646
BytebuddyAgent [candidate] (656.42 ms) : 0, 656420
AgentMeter [baseline] (11.744 ms) : 0, 11744
AgentMeter [candidate] (11.723 ms) : 0, 11723
GlobalTracer [baseline] (257.717 ms) : 0, 257717
GlobalTracer [candidate] (257.868 ms) : 0, 257868
AppSec [baseline] (167.198 ms) : 0, 167198
AppSec [candidate] (168.045 ms) : 0, 168045
Debugger [baseline] (66.911 ms) : 0, 66911
Debugger [candidate] (66.298 ms) : 0, 66298
Remote Config [baseline] (662.385 µs) : 0, 662
Remote Config [candidate] (667.563 µs) : 0, 668
Telemetry [baseline] (9.115 ms) : 0, 9115
Telemetry [candidate] (9.153 ms) : 0, 9153
Flare Poller [baseline] (3.644 ms) : 0, 3644
Flare Poller [candidate] (3.635 ms) : 0, 3635
IAST [baseline] (25.117 ms) : 0, 25117
IAST [candidate] (25.177 ms) : 0, 25177
section iast
crashtracking [baseline] (1.184 ms) : 0, 1184
crashtracking [candidate] (1.178 ms) : 0, 1178
BytebuddyAgent [baseline] (798.627 ms) : 0, 798627
BytebuddyAgent [candidate] (795.111 ms) : 0, 795111
AgentMeter [baseline] (11.111 ms) : 0, 11111
AgentMeter [candidate] (11.139 ms) : 0, 11139
GlobalTracer [baseline] (247.553 ms) : 0, 247553
GlobalTracer [candidate] (247.36 ms) : 0, 247360
AppSec [baseline] (33.569 ms) : 0, 33569
AppSec [candidate] (33.773 ms) : 0, 33773
Debugger [baseline] (66.102 ms) : 0, 66102
Debugger [candidate] (66.359 ms) : 0, 66359
Remote Config [baseline] (555.564 µs) : 0, 556
Remote Config [candidate] (543.102 µs) : 0, 543
Telemetry [baseline] (8.809 ms) : 0, 8809
Telemetry [candidate] (8.769 ms) : 0, 8769
Flare Poller [baseline] (3.493 ms) : 0, 3493
Flare Poller [candidate] (3.496 ms) : 0, 3496
IAST [baseline] (26.956 ms) : 0, 26956
IAST [candidate] (26.875 ms) : 0, 26875
section profiling
ProfilingAgent [baseline] (99.096 ms) : 0, 99096
ProfilingAgent [candidate] (98.61 ms) : 0, 98610
crashtracking [baseline] (1.204 ms) : 0, 1204
crashtracking [candidate] (1.208 ms) : 0, 1208
BytebuddyAgent [baseline] (682.553 ms) : 0, 682553
BytebuddyAgent [candidate] (679.575 ms) : 0, 679575
AgentMeter [baseline] (8.648 ms) : 0, 8648
AgentMeter [candidate] (8.621 ms) : 0, 8621
GlobalTracer [baseline] (215.806 ms) : 0, 215806
GlobalTracer [candidate] (215.806 ms) : 0, 215806
AppSec [baseline] (32.511 ms) : 0, 32511
AppSec [candidate] (32.447 ms) : 0, 32447
Debugger [baseline] (67.21 ms) : 0, 67210
Debugger [candidate] (67.14 ms) : 0, 67140
Remote Config [baseline] (588.188 µs) : 0, 588
Remote Config [candidate] (594.5 µs) : 0, 595
Telemetry [baseline] (8.818 ms) : 0, 8818
Telemetry [candidate] (8.761 ms) : 0, 8761
Flare Poller [baseline] (3.752 ms) : 0, 3752
Flare Poller [candidate] (3.738 ms) : 0, 3738
Profiling [baseline] (99.665 ms) : 0, 99665
Profiling [candidate] (99.176 ms) : 0, 99176
LoadParameters
See matching parameters
SummaryFound 2 performance improvements and 1 performance regressions! Performance is the same for 17 metrics, 16 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~46230e2bb1, baseline=1.59.0-SNAPSHOT~f2f109ef18
dateFormat X
axisFormat %s
section baseline
no_agent (1.179 ms) : 1168, 1190
. : milestone, 1179,
iast (3.128 ms) : 3090, 3167
. : milestone, 3128,
iast_FULL (5.822 ms) : 5764, 5880
. : milestone, 5822,
iast_GLOBAL (3.413 ms) : 3369, 3457
. : milestone, 3413,
profiling (2.1 ms) : 2081, 2119
. : milestone, 2100,
tracing (1.839 ms) : 1823, 1854
. : milestone, 1839,
section candidate
no_agent (1.187 ms) : 1176, 1198
. : milestone, 1187,
iast (3.218 ms) : 3175, 3261
. : milestone, 3218,
iast_FULL (5.894 ms) : 5835, 5954
. : milestone, 5894,
iast_GLOBAL (3.421 ms) : 3374, 3467
. : milestone, 3421,
profiling (2.027 ms) : 2009, 2044
. : milestone, 2027,
tracing (1.864 ms) : 1847, 1880
. : milestone, 1864,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~46230e2bb1, baseline=1.59.0-SNAPSHOT~f2f109ef18
dateFormat X
axisFormat %s
section baseline
no_agent (19.245 ms) : 19043, 19446
. : milestone, 19245,
appsec (18.752 ms) : 18565, 18938
. : milestone, 18752,
code_origins (17.587 ms) : 17409, 17765
. : milestone, 17587,
iast (17.689 ms) : 17510, 17869
. : milestone, 17689,
profiling (19.542 ms) : 19345, 19740
. : milestone, 19542,
tracing (18.018 ms) : 17836, 18200
. : milestone, 18018,
section candidate
no_agent (19.123 ms) : 18930, 19317
. : milestone, 19123,
appsec (18.893 ms) : 18702, 19084
. : milestone, 18893,
code_origins (18.765 ms) : 18571, 18959
. : milestone, 18765,
iast (17.45 ms) : 17276, 17625
. : milestone, 17450,
profiling (18.232 ms) : 18051, 18413
. : milestone, 18232,
tracing (17.623 ms) : 17449, 17797
. : milestone, 17623,
DacapoParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 1 unstable metrics.
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~46230e2bb1, baseline=1.59.0-SNAPSHOT~f2f109ef18
dateFormat X
axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1488
. : milestone, 1476,
appsec (3.734 ms) : 3515, 3953
. : milestone, 3734,
iast (2.261 ms) : 2191, 2330
. : milestone, 2261,
iast_GLOBAL (2.295 ms) : 2225, 2365
. : milestone, 2295,
profiling (2.508 ms) : 2344, 2673
. : milestone, 2508,
tracing (2.053 ms) : 2000, 2107
. : milestone, 2053,
section candidate
no_agent (1.475 ms) : 1464, 1487
. : milestone, 1475,
appsec (2.494 ms) : 2439, 2549
. : milestone, 2494,
iast (2.259 ms) : 2189, 2329
. : milestone, 2259,
iast_GLOBAL (2.301 ms) : 2232, 2370
. : milestone, 2301,
profiling (2.076 ms) : 2021, 2131
. : milestone, 2076,
tracing (2.058 ms) : 2004, 2112
. : milestone, 2058,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~46230e2bb1, baseline=1.59.0-SNAPSHOT~f2f109ef18
dateFormat X
axisFormat %s
section baseline
no_agent (15.722 s) : 15722000, 15722000
. : milestone, 15722000,
appsec (14.657 s) : 14657000, 14657000
. : milestone, 14657000,
iast (18.243 s) : 18243000, 18243000
. : milestone, 18243000,
iast_GLOBAL (18.089 s) : 18089000, 18089000
. : milestone, 18089000,
profiling (14.958 s) : 14958000, 14958000
. : milestone, 14958000,
tracing (14.776 s) : 14776000, 14776000
. : milestone, 14776000,
section candidate
no_agent (15.244 s) : 15244000, 15244000
. : milestone, 15244000,
appsec (15.093 s) : 15093000, 15093000
. : milestone, 15093000,
iast (18.194 s) : 18194000, 18194000
. : milestone, 18194000,
iast_GLOBAL (17.93 s) : 17930000, 17930000
. : milestone, 17930000,
profiling (14.941 s) : 14941000, 14941000
. : milestone, 14941000,
tracing (14.826 s) : 14826000, 14826000
. : milestone, 14826000,
|
What Does This Do
It skips the mentioned test on Oracle JDK 8
Motivation
For some reason, the generated JFR file can not be read back by the JMC parser on the runners used to run the Oracle JDK 8 tests. When I download the same file and open it locally, it is fine.
The reason is not really clear and for the sakes of unblocking master failures, let's just skip the test on Oracle JDK 8 for now.
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any useful labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issue