Skip to content

Baggage E2E "working" #8699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 13 commits into
base: master
Choose a base branch
from
Draft

Baggage E2E "working" #8699

wants to merge 13 commits into from
+663 −239

Conversation

mhlidd
Copy link
Contributor

@mhlidd mhlidd commented Apr 15, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@mhlidd mhlidd added the tag: do not merge Do not merge changes label Apr 15, 2025
@pr-commenter
Copy link

pr-commenter bot commented Apr 15, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/enable_baggage_end_to_end
git_commit_date 1745582823 1745596847
git_commit_sha 75634fe d21423e
release_version 1.49.0-SNAPSHOT~75634fe423 1.48.0-SNAPSHOT~d21423e3e8
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1745599899 1745599899
ci_job_id 913220196 913220196
ci_pipeline_id 63363736 63363736
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-uvgw7qef-project-304-concurrent-0-hbh8w131 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-uvgw7qef-project-304-concurrent-0-hbh8w131 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 9 performance improvements and 24 performance regressions! Performance is the same for 31 metrics, 7 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent.start worse
[+38.125ms; +45.531ms] or [+3.354%; +4.006%]		 1.178s 1.137s
scenario:startup:insecure-bank:iast:BytebuddyAgent worse
[+44.196ms; +49.189ms] or [+5.593%; +6.225%]		 836.837ms 790.144ms
scenario:startup:insecure-bank:iast:Debugger better
[-1.801ms; -1.655ms] or [-30.921%; -28.408%]		 4.097ms 5.825ms
scenario:startup:insecure-bank:iast:Remote Config worse
[+13.959µs; +46.208µs] or [+2.378%; +7.871%]		 617.124µs 587.040µs
scenario:startup:insecure-bank:iast:Telemetry worse
[+713.600µs; +937.919µs] or [+9.117%; +11.982%]		 8.653ms 7.827ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent.start worse
[+24.951ms; +70.690ms] or [+2.183%; +6.184%]		 1.191s 1.143s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:BytebuddyAgent worse
[+33.960ms; +69.037ms] or [+4.274%; +8.688%]		 846.123ms 794.624ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Debugger better
[-1.846ms; -1.516ms] or [-31.543%; -25.902%]		 4.172ms 5.853ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Telemetry worse
[+683.033µs; +1200.871µs] or [+8.628%; +15.169%]		 8.859ms 7.917ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent.start worse
[+42.390ms; +49.181ms] or [+3.739%; +4.337%]		 1.180s 1.134s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:BytebuddyAgent worse
[+46.604ms; +52.386ms] or [+5.921%; +6.656%]		 836.574ms 787.079ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Debugger better
[-1.877ms; -1.525ms] or [-31.546%; -25.627%]		 4.248ms 5.949ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Telemetry worse
[+0.740ms; +1.363ms] or [+9.547%; +17.579%]		 8.807ms 7.756ms
scenario:startup:insecure-bank:tracing:Agent.start worse
[+27.509ms; +54.929ms] or [+2.714%; +5.419%]		 1.055s 1.014s
scenario:startup:insecure-bank:tracing:BytebuddyAgent worse
[+34.321ms; +54.445ms] or [+5.081%; +8.061%]		 719.825ms 675.442ms
scenario:startup:insecure-bank:tracing:Debugger better
[-1.927ms; -1.640ms] or [-30.904%; -26.297%]		 4.453ms 6.237ms
scenario:startup:petclinic:appsec:Agent.start worse
[+28.444ms; +58.187ms] or [+2.464%; +5.040%]		 1.198s 1.154s
scenario:startup:petclinic:appsec:BytebuddyAgent worse
[+39.456ms; +57.311ms] or [+5.702%; +8.283%]		 740.301ms 691.918ms
scenario:startup:petclinic:appsec:Debugger better
[-1.732ms; -1.584ms] or [-29.211%; -26.719%]		 4.271ms 5.929ms
scenario:startup:petclinic:iast:Agent.start worse
[+38.530ms; +48.506ms] or [+3.386%; +4.263%]		 1.181s 1.138s
scenario:startup:petclinic:iast:BytebuddyAgent worse
[+44.801ms; +52.047ms] or [+5.667%; +6.584%]		 838.911ms 790.487ms
scenario:startup:petclinic:iast:Debugger better
[-1.841ms; -1.654ms] or [-31.334%; -28.140%]		 4.129ms 5.877ms
scenario:startup:petclinic:iast:Telemetry worse
[+604.561µs; +882.198µs] or [+7.624%; +11.125%]		 8.673ms 7.930ms
scenario:startup:petclinic:profiling:ProfilingAgent worse
[+3.950ms; +7.942ms] or [+4.052%; +8.147%]		 103.435ms 97.489ms
scenario:startup:petclinic:profiling:BytebuddyAgent worse
[+46.636ms; +61.612ms] or [+7.051%; +9.315%]		 715.560ms 661.436ms
scenario:startup:petclinic:profiling:GlobalTracer better
[-25.811ms; -18.164ms] or [-6.892%; -4.850%]		 352.547ms 374.534ms
scenario:startup:petclinic:profiling:Debugger better
[-1.926ms; -1.750ms] or [-31.401%; -28.527%]		 4.296ms 6.134ms
scenario:startup:petclinic:profiling:Remote Config worse
[+44.670µs; +90.524µs] or [+6.840%; +13.861%]		 720.676µs 653.079µs
scenario:startup:petclinic:profiling:Telemetry worse
[+809.666µs; +1140.127µs] or [+10.047%; +14.148%]		 9.033ms 8.059ms
scenario:startup:petclinic:profiling:Profiling worse
[+3.952ms; +7.945ms] or [+4.053%; +8.148%]		 103.462ms 97.513ms
scenario:startup:petclinic:tracing:Agent.start worse
[+28.596ms; +58.265ms] or [+2.831%; +5.768%]		 1.053s 1.010s
scenario:startup:petclinic:tracing:BytebuddyAgent worse
[+35.753ms; +55.597ms] or [+5.312%; +8.260%]		 718.728ms 673.053ms
scenario:startup:petclinic:tracing:Debugger better
[-1.896ms; -1.635ms] or [-30.604%; -26.399%]		 4.429ms 6.194ms
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~d21423e3e8, baseline=1.49.0-SNAPSHOT~75634fe423

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.014 s) : 0, 1013693
Total [baseline] (8.697 s) : 0, 8696990
Agent [candidate] (1.055 s) : 0, 1054912
Total [candidate] (8.683 s) : 0, 8682620
section iast
Agent [baseline] (1.137 s) : 0, 1136605
Total [baseline] (9.246 s) : 0, 9245975
Agent [candidate] (1.178 s) : 0, 1178433
Total [candidate] (9.284 s) : 0, 9284435
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.143 s) : 0, 1143193
Total [baseline] (9.232 s) : 0, 9232187
Agent [candidate] (1.191 s) : 0, 1191013
Total [candidate] (9.276 s) : 0, 9276315
section iast_TELEMETRY_OFF
Agent [baseline] (1.134 s) : 0, 1133872
Total [baseline] (9.234 s) : 0, 9234306
Agent [candidate] (1.18 s) : 0, 1179658
Total [candidate] (9.336 s) : 0, 9335962
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.014 s -
Agent iast 1.137 s 122.912 ms (12.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.143 s 129.5 ms (12.8%)
Agent iast_TELEMETRY_OFF 1.134 s 120.179 ms (11.9%)
Total tracing 8.697 s -
Total iast 9.246 s 548.985 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.232 s 535.197 ms (6.2%)
Total iast_TELEMETRY_OFF 9.234 s 537.316 ms (6.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.055 s -
Agent iast 1.178 s 123.521 ms (11.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.191 s 136.101 ms (12.9%)
Agent iast_TELEMETRY_OFF 1.18 s 124.746 ms (11.8%)
Total tracing 8.683 s -
Total iast 9.284 s 601.815 ms (6.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.276 s 593.695 ms (6.8%)
Total iast_TELEMETRY_OFF 9.336 s 653.342 ms (7.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~d21423e3e8, baseline=1.49.0-SNAPSHOT~75634fe423

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (675.442 ms) : 0, 675442
BytebuddyAgent [candidate] (719.825 ms) : 0, 719825
GlobalTracer [baseline] (241.469 ms) : 0, 241469
GlobalTracer [candidate] (239.607 ms) : 0, 239607
AppSec [baseline] (55.121 ms) : 0, 55121
AppSec [candidate] (55.179 ms) : 0, 55179
Debugger [baseline] (6.237 ms) : 0, 6237
Debugger [candidate] (4.453 ms) : 0, 4453
Remote Config [baseline] (728.793 µs) : 0, 729
Remote Config [candidate] (697.702 µs) : 0, 698
Telemetry [baseline] (11.11 ms) : 0, 11110
Telemetry [candidate] (14.505 ms) : 0, 14505
section iast
BytebuddyAgent [baseline] (790.144 ms) : 0, 790144
BytebuddyAgent [candidate] (836.837 ms) : 0, 836837
GlobalTracer [baseline] (229.954 ms) : 0, 229954
GlobalTracer [candidate] (229.391 ms) : 0, 229391
IAST [baseline] (22.579 ms) : 0, 22579
IAST [candidate] (22.729 ms) : 0, 22729
AppSec [baseline] (56.302 ms) : 0, 56302
AppSec [candidate] (55.706 ms) : 0, 55706
Debugger [baseline] (5.825 ms) : 0, 5825
Debugger [candidate] (4.097 ms) : 0, 4097
Remote Config [baseline] (587.04 µs) : 0, 587
Remote Config [candidate] (617.124 µs) : 0, 617
Telemetry [baseline] (7.827 ms) : 0, 7827
Telemetry [candidate] (8.653 ms) : 0, 8653
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (794.624 ms) : 0, 794624
BytebuddyAgent [candidate] (846.123 ms) : 0, 846123
GlobalTracer [baseline] (231.41 ms) : 0, 231410
GlobalTracer [candidate] (231.304 ms) : 0, 231304
IAST [baseline] (22.942 ms) : 0, 22942
IAST [candidate] (23.052 ms) : 0, 23052
AppSec [baseline] (56.365 ms) : 0, 56365
AppSec [candidate] (56.189 ms) : 0, 56189
Debugger [baseline] (5.853 ms) : 0, 5853
Debugger [candidate] (4.172 ms) : 0, 4172
Remote Config [baseline] (605.872 µs) : 0, 606
Remote Config [candidate] (627.228 µs) : 0, 627
Telemetry [baseline] (7.917 ms) : 0, 7917
Telemetry [candidate] (8.859 ms) : 0, 8859
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (787.079 ms) : 0, 787079
BytebuddyAgent [candidate] (836.574 ms) : 0, 836574
GlobalTracer [baseline] (230.117 ms) : 0, 230117
GlobalTracer [candidate] (229.674 ms) : 0, 229674
IAST [baseline] (22.379 ms) : 0, 22379
IAST [candidate] (23.525 ms) : 0, 23525
AppSec [baseline] (56.632 ms) : 0, 56632
AppSec [candidate] (55.736 ms) : 0, 55736
Debugger [baseline] (5.949 ms) : 0, 5949
Debugger [candidate] (4.248 ms) : 0, 4248
Remote Config [baseline] (622.225 µs) : 0, 622
Remote Config [candidate] (628.124 µs) : 0, 628
Telemetry [baseline] (7.756 ms) : 0, 7756
Telemetry [candidate] (8.807 ms) : 0, 8807
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~d21423e3e8, baseline=1.49.0-SNAPSHOT~75634fe423

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.01 s) : 0, 1010069
Total [baseline] (10.511 s) : 0, 10511223
Agent [candidate] (1.053 s) : 0, 1053500
Total [candidate] (10.452 s) : 0, 10452068
section appsec
Agent [baseline] (1.154 s) : 0, 1154411
Total [baseline] (10.743 s) : 0, 10742800
Agent [candidate] (1.198 s) : 0, 1197727
Total [candidate] (10.849 s) : 0, 10848745
section iast
Agent [baseline] (1.138 s) : 0, 1137898
Total [baseline] (10.921 s) : 0, 10920588
Agent [candidate] (1.181 s) : 0, 1181416
Total [candidate] (11.054 s) : 0, 11054439
section profiling
Agent [baseline] (1.253 s) : 0, 1252739
Total [baseline] (10.86 s) : 0, 10860162
Agent [candidate] (1.285 s) : 0, 1284850
Total [candidate] (10.947 s) : 0, 10947283
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.01 s -
Agent appsec 1.154 s 144.341 ms (14.3%)
Agent iast 1.138 s 127.829 ms (12.7%)
Agent profiling 1.253 s 242.669 ms (24.0%)
Total tracing 10.511 s -
Total appsec 10.743 s 231.577 ms (2.2%)
Total iast 10.921 s 409.365 ms (3.9%)
Total profiling 10.86 s 348.939 ms (3.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.053 s -
Agent appsec 1.198 s 144.227 ms (13.7%)
Agent iast 1.181 s 127.916 ms (12.1%)
Agent profiling 1.285 s 231.35 ms (22.0%)
Total tracing 10.452 s -
Total appsec 10.849 s 396.677 ms (3.8%)
Total iast 11.054 s 602.371 ms (5.8%)
Total profiling 10.947 s 495.216 ms (4.7%) 
gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~d21423e3e8, baseline=1.49.0-SNAPSHOT~75634fe423

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (673.053 ms) : 0, 673053
BytebuddyAgent [candidate] (718.728 ms) : 0, 718728
GlobalTracer [baseline] (241.05 ms) : 0, 241050
GlobalTracer [candidate] (239.51 ms) : 0, 239510
AppSec [baseline] (54.91 ms) : 0, 54910
AppSec [candidate] (54.507 ms) : 0, 54507
Debugger [baseline] (6.194 ms) : 0, 6194
Debugger [candidate] (4.429 ms) : 0, 4429
Remote Config [baseline] (724.516 µs) : 0, 725
Remote Config [candidate] (695.416 µs) : 0, 695
Telemetry [baseline] (10.535 ms) : 0, 10535
Telemetry [candidate] (15.185 ms) : 0, 15185
section appsec
BytebuddyAgent [baseline] (691.918 ms) : 0, 691918
BytebuddyAgent [candidate] (740.301 ms) : 0, 740301
GlobalTracer [baseline] (237.712 ms) : 0, 237712
GlobalTracer [candidate] (236.653 ms) : 0, 236653
AppSec [baseline] (176.266 ms) : 0, 176266
AppSec [candidate] (175.901 ms) : 0, 175901
Debugger [baseline] (5.929 ms) : 0, 5929
Debugger [candidate] (4.271 ms) : 0, 4271
Remote Config [baseline] (640.295 µs) : 0, 640
Remote Config [candidate] (642.848 µs) : 0, 643
Telemetry [baseline] (7.433 ms) : 0, 7433
Telemetry [candidate] (8.568 ms) : 0, 8568
IAST [baseline] (21.927 ms) : 0, 21927
IAST [candidate] (21.773 ms) : 0, 21773
section iast
BytebuddyAgent [baseline] (790.487 ms) : 0, 790487
BytebuddyAgent [candidate] (838.911 ms) : 0, 838911
GlobalTracer [baseline] (230.377 ms) : 0, 230377
GlobalTracer [candidate] (229.803 ms) : 0, 229803
AppSec [baseline] (56.594 ms) : 0, 56594
AppSec [candidate] (55.874 ms) : 0, 55874
Debugger [baseline] (5.877 ms) : 0, 5877
Debugger [candidate] (4.129 ms) : 0, 4129
Remote Config [baseline] (603.139 µs) : 0, 603
Remote Config [candidate] (635.256 µs) : 0, 635
Telemetry [baseline] (7.93 ms) : 0, 7930
Telemetry [candidate] (8.673 ms) : 0, 8673
IAST [baseline] (22.647 ms) : 0, 22647
IAST [candidate] (22.87 ms) : 0, 22870
section profiling
ProfilingAgent [baseline] (97.489 ms) : 0, 97489
ProfilingAgent [candidate] (103.435 ms) : 0, 103435
BytebuddyAgent [baseline] (661.436 ms) : 0, 661436
BytebuddyAgent [candidate] (715.56 ms) : 0, 715560
GlobalTracer [baseline] (374.534 ms) : 0, 374534
GlobalTracer [candidate] (352.547 ms) : 0, 352547
AppSec [baseline] (54.285 ms) : 0, 54285
AppSec [candidate] (53.602 ms) : 0, 53602
Debugger [baseline] (6.134 ms) : 0, 6134
Debugger [candidate] (4.296 ms) : 0, 4296
Remote Config [baseline] (653.079 µs) : 0, 653
Remote Config [candidate] (720.676 µs) : 0, 721
Telemetry [baseline] (8.059 ms) : 0, 8059
Telemetry [candidate] (9.033 ms) : 0, 9033
Profiling [baseline] (97.513 ms) : 0, 97513
Profiling [candidate] (103.462 ms) : 0, 103462
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-04-25T16:21:44 2025-04-25T16:29:33
git_branch master mhlidd/enable_baggage_end_to_end
git_commit_date 1745582823 1745596847
git_commit_sha 75634fe d21423e
release_version 1.49.0-SNAPSHOT~75634fe423 1.48.0-SNAPSHOT~d21423e3e8
start_time 2025-04-25T16:21:29 2025-04-25T16:29:18
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1745598975 1745598975
ci_job_id 913220199 913220199
ci_pipeline_id 63363736 63363736
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-nsrtnna7-project-304-concurrent-0-1a7hef5e 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-nsrtnna7-project-304-concurrent-0-1a7hef5e 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 18 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling better
[-89.771µs; -37.226µs] or [-5.617%; -2.329%]		 unstable
[-411.240op/s; +639.160op/s] or [-13.879%; +21.572%]		 1.535ms 3076.923op/s 1.598ms 2962.963op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~d21423e3e8, baseline=1.49.0-SNAPSHOT~75634fe423
    dateFormat X
    axisFormat %s
section baseline
no_agent (385.024 µs) : 365, 405
.   : milestone, 385,
iast (527.443 µs) : 504, 551
.   : milestone, 527,
iast_FULL (747.053 µs) : 723, 771
.   : milestone, 747,
iast_GLOBAL (569.216 µs) : 546, 592
.   : milestone, 569,
iast_HARDCODED_SECRET_DISABLED (533.481 µs) : 511, 556
.   : milestone, 533,
iast_INACTIVE (474.304 µs) : 452, 497
.   : milestone, 474,
iast_TELEMETRY_OFF (527.459 µs) : 504, 551
.   : milestone, 527,
tracing (467.9 µs) : 445, 490
.   : milestone, 468,
section candidate
no_agent (388.714 µs) : 369, 408
.   : milestone, 389,
iast (528.765 µs) : 507, 551
.   : milestone, 529,
iast_FULL (740.433 µs) : 718, 762
.   : milestone, 740,
iast_GLOBAL (569.219 µs) : 548, 591
.   : milestone, 569,
iast_HARDCODED_SECRET_DISABLED (522.842 µs) : 501, 545
.   : milestone, 523,
iast_INACTIVE (473.957 µs) : 452, 496
.   : milestone, 474,
iast_TELEMETRY_OFF (514.857 µs) : 493, 537
.   : milestone, 515,
tracing (476.206 µs) : 454, 498
.   : milestone, 476,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 385.024 µs [365.119 µs, 404.928 µs] -
iast 527.443 µs [504.368 µs, 550.517 µs] 142.419 µs (37.0%)
iast_FULL 747.053 µs [723.235 µs, 770.87 µs] 362.029 µs (94.0%)
iast_GLOBAL 569.216 µs [546.173 µs, 592.26 µs] 184.193 µs (47.8%)
iast_HARDCODED_SECRET_DISABLED 533.481 µs [510.863 µs, 556.1 µs] 148.457 µs (38.6%)
iast_INACTIVE 474.304 µs [451.87 µs, 496.737 µs] 89.28 µs (23.2%)
iast_TELEMETRY_OFF 527.459 µs [504.033 µs, 550.886 µs] 142.436 µs (37.0%)
tracing 467.9 µs [445.412 µs, 490.388 µs] 82.876 µs (21.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 388.714 µs [369.058 µs, 408.369 µs] -
iast 528.765 µs [506.797 µs, 550.733 µs] 140.052 µs (36.0%)
iast_FULL 740.433 µs [718.475 µs, 762.391 µs] 351.72 µs (90.5%)
iast_GLOBAL 569.219 µs [547.512 µs, 590.925 µs] 180.505 µs (46.4%)
iast_HARDCODED_SECRET_DISABLED 522.842 µs [501.01 µs, 544.675 µs] 134.129 µs (34.5%)
iast_INACTIVE 473.957 µs [451.639 µs, 496.274 µs] 85.243 µs (21.9%)
iast_TELEMETRY_OFF 514.857 µs [492.842 µs, 536.872 µs] 126.143 µs (32.5%)
tracing 476.206 µs [453.92 µs, 498.492 µs] 87.493 µs (22.5%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~d21423e3e8, baseline=1.49.0-SNAPSHOT~75634fe423
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.37 ms) : 1351, 1390
.   : milestone, 1370,
appsec (1.76 ms) : 1737, 1784
.   : milestone, 1760,
appsec_no_iast (1.742 ms) : 1718, 1765
.   : milestone, 1742,
code_origins (1.694 ms) : 1667, 1721
.   : milestone, 1694,
iast (1.533 ms) : 1509, 1558
.   : milestone, 1533,
profiling (1.598 ms) : 1573, 1624
.   : milestone, 1598,
tracing (1.496 ms) : 1470, 1522
.   : milestone, 1496,
section candidate
no_agent (1.361 ms) : 1341, 1381
.   : milestone, 1361,
appsec (1.754 ms) : 1731, 1778
.   : milestone, 1754,
appsec_no_iast (1.735 ms) : 1712, 1759
.   : milestone, 1735,
code_origins (1.694 ms) : 1666, 1721
.   : milestone, 1694,
iast (1.536 ms) : 1512, 1560
.   : milestone, 1536,
profiling (1.535 ms) : 1511, 1558
.   : milestone, 1535,
tracing (1.518 ms) : 1494, 1541
.   : milestone, 1518,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.37 ms [1.351 ms, 1.39 ms] -
appsec 1.76 ms [1.737 ms, 1.784 ms] 389.762 µs (28.4%)
appsec_no_iast 1.742 ms [1.718 ms, 1.765 ms] 371.371 µs (27.1%)
code_origins 1.694 ms [1.667 ms, 1.721 ms] 323.582 µs (23.6%)
iast 1.533 ms [1.509 ms, 1.558 ms] 162.817 µs (11.9%)
profiling 1.598 ms [1.573 ms, 1.624 ms] 227.612 µs (16.6%)
tracing 1.496 ms [1.47 ms, 1.522 ms] 125.663 µs (9.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.361 ms [1.341 ms, 1.381 ms] -
appsec 1.754 ms [1.731 ms, 1.778 ms] 392.761 µs (28.8%)
appsec_no_iast 1.735 ms [1.712 ms, 1.759 ms] 373.872 µs (27.5%)
code_origins 1.694 ms [1.666 ms, 1.721 ms] 332.244 µs (24.4%)
iast 1.536 ms [1.512 ms, 1.56 ms] 174.738 µs (12.8%)
profiling 1.535 ms [1.511 ms, 1.558 ms] 173.175 µs (12.7%)
tracing 1.518 ms [1.494 ms, 1.541 ms] 156.223 µs (11.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/enable_baggage_end_to_end
git_commit_date 1745582823 1745596847
git_commit_sha 75634fe d21423e
release_version 1.49.0-SNAPSHOT~75634fe423 1.48.0-SNAPSHOT~d21423e3e8
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1745599475 1745599475
ci_job_id 913220202 913220202
ci_pipeline_id 63363736 63363736
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-uvgw7qef-project-304-concurrent-1-8m7utha2 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-uvgw7qef-project-304-concurrent-1-8m7utha2 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~d21423e3e8, baseline=1.49.0-SNAPSHOT~75634fe423
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (2.36 ms) : 2314, 2406
.   : milestone, 2360,
iast (2.141 ms) : 2082, 2199
.   : milestone, 2141,
iast_GLOBAL (2.195 ms) : 2135, 2254
.   : milestone, 2195,
profiling (2.009 ms) : 1961, 2057
.   : milestone, 2009,
tracing (1.984 ms) : 1939, 2029
.   : milestone, 1984,
section candidate
no_agent (1.471 ms) : 1459, 1482
.   : milestone, 1471,
appsec (2.376 ms) : 2328, 2425
.   : milestone, 2376,
iast (2.151 ms) : 2092, 2210
.   : milestone, 2151,
iast_GLOBAL (2.189 ms) : 2131, 2248
.   : milestone, 2189,
profiling (2.013 ms) : 1965, 2060
.   : milestone, 2013,
tracing (1.973 ms) : 1928, 2018
.   : milestone, 1973,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.458 ms, 1.481 ms] -
appsec 2.36 ms [2.314 ms, 2.406 ms] 890.711 µs (60.6%)
iast 2.141 ms [2.082 ms, 2.199 ms] 671.35 µs (45.7%)
iast_GLOBAL 2.195 ms [2.135 ms, 2.254 ms] 725.213 µs (49.4%)
profiling 2.009 ms [1.961 ms, 2.057 ms] 539.858 µs (36.7%)
tracing 1.984 ms [1.939 ms, 2.029 ms] 514.536 µs (35.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.471 ms [1.459 ms, 1.482 ms] -
appsec 2.376 ms [2.328 ms, 2.425 ms] 905.73 µs (61.6%)
iast 2.151 ms [2.092 ms, 2.21 ms] 680.194 µs (46.2%)
iast_GLOBAL 2.189 ms [2.131 ms, 2.248 ms] 718.729 µs (48.9%)
profiling 2.013 ms [1.965 ms, 2.06 ms] 541.935 µs (36.8%)
tracing 1.973 ms [1.928 ms, 2.018 ms] 502.452 µs (34.2%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~d21423e3e8, baseline=1.49.0-SNAPSHOT~75634fe423
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.992 s) : 14992000, 14992000
.   : milestone, 14992000,
appsec (14.822 s) : 14822000, 14822000
.   : milestone, 14822000,
iast (18.542 s) : 18542000, 18542000
.   : milestone, 18542000,
iast_GLOBAL (18.207 s) : 18207000, 18207000
.   : milestone, 18207000,
profiling (15.114 s) : 15114000, 15114000
.   : milestone, 15114000,
tracing (14.927 s) : 14927000, 14927000
.   : milestone, 14927000,
section candidate
no_agent (15.247 s) : 15247000, 15247000
.   : milestone, 15247000,
appsec (15.005 s) : 15005000, 15005000
.   : milestone, 15005000,
iast (18.899 s) : 18899000, 18899000
.   : milestone, 18899000,
iast_GLOBAL (18.254 s) : 18254000, 18254000
.   : milestone, 18254000,
profiling (15.127 s) : 15127000, 15127000
.   : milestone, 15127000,
tracing (14.888 s) : 14888000, 14888000
.   : milestone, 14888000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.992 s [14.992 s, 14.992 s] -
appsec 14.822 s [14.822 s, 14.822 s] -170.0 ms (-1.1%)
iast 18.542 s [18.542 s, 18.542 s] 3.55 s (23.7%)
iast_GLOBAL 18.207 s [18.207 s, 18.207 s] 3.215 s (21.4%)
profiling 15.114 s [15.114 s, 15.114 s] 122.0 ms (0.8%)
tracing 14.927 s [14.927 s, 14.927 s] -65.0 ms (-0.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.247 s [15.247 s, 15.247 s] -
appsec 15.005 s [15.005 s, 15.005 s] -242.0 ms (-1.6%)
iast 18.899 s [18.899 s, 18.899 s] 3.652 s (24.0%)
iast_GLOBAL 18.254 s [18.254 s, 18.254 s] 3.007 s (19.7%)
profiling 15.127 s [15.127 s, 15.127 s] -120.0 ms (-0.8%)
tracing 14.888 s [14.888 s, 14.888 s] -359.0 ms (-2.4%)

@pr-commenter
Copy link

pr-commenter bot commented Apr 16, 2025 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/enable_baggage_end_to_end
git_commit_date 1743601182 1745596847
git_commit_sha 089d1cc d21423e
See matching parameters
Baseline Candidate
ci_job_date 1745598528 1745598528
ci_job_id 913220207 913220207
ci_pipeline_id 63363736 63363736
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter
Copy link

pr-commenter bot commented Apr 16, 2025 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/enable_baggage_end_to_end
git_commit_date 1743601182 1745596847
git_commit_sha 089d1cc d21423e
See matching parameters
Baseline Candidate
ci_job_date 1745598556 1745598556
ci_job_id 913220209 913220209
ci_pipeline_id 63363736 63363736
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
tag: do not merge Do not merge changes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@mhlidd