Skip to content

Remove muzzle from CircleCI #8703

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Remove muzzle from CircleCI #8703

wants to merge 1 commit into from

Conversation

randomanderson
Copy link
Contributor

What Does This Do

Removes muzzle jobs from CircleCI

Motivation

Eventually, all jobs will be migrated from CircleCI to GitLab. Muzzle jobs in Gitlab have comparable failure rates of 1-3% with a flakyness of 0%. The flakyness indicates that these failures were actual failures and not issues with the CI system.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@randomanderson randomanderson added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Apr 16, 2025
@randomanderson randomanderson requested a review from a team as a code owner April 16, 2025 16:56
@randomanderson randomanderson requested a review from ygree April 16, 2025 16:56
@pr-commenter
Copy link

pr-commenter bot commented Apr 16, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master landerson/remove-circleci-muzzle
git_commit_date 1744612276 1744821916
git_commit_sha b00c24e 12273ce
release_version 1.49.0-SNAPSHOT~b00c24e26a 1.49.0-SNAPSHOT~12273ce178
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1744825080 1744825080
ci_job_id 899981209 899981209
ci_pipeline_id 62397027 62397027
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-tymfbhrg-project-304-concurrent-0-hl3zr8na 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-tymfbhrg-project-304-concurrent-0-hl3zr8na 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 61 metrics, 10 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~12273ce178, baseline=1.49.0-SNAPSHOT~b00c24e26a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1056988
Total [baseline] (8.684 s) : 0, 8683864
Agent [candidate] (1.061 s) : 0, 1061118
Total [candidate] (8.687 s) : 0, 8686737
section iast
Agent [baseline] (1.184 s) : 0, 1184455
Total [baseline] (9.314 s) : 0, 9314274
Agent [candidate] (1.186 s) : 0, 1186264
Total [candidate] (9.249 s) : 0, 9248976
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.187 s) : 0, 1187311
Total [baseline] (9.298 s) : 0, 9298368
Agent [candidate] (1.187 s) : 0, 1186856
Total [candidate] (9.259 s) : 0, 9258860
section iast_TELEMETRY_OFF
Agent [baseline] (1.186 s) : 0, 1186371
Total [baseline] (9.31 s) : 0, 9309964
Agent [candidate] (1.182 s) : 0, 1181661
Total [candidate] (9.282 s) : 0, 9281744
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent iast 1.184 s 127.467 ms (12.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.187 s 130.323 ms (12.3%)
Agent iast_TELEMETRY_OFF 1.186 s 129.384 ms (12.2%)
Total tracing 8.684 s -
Total iast 9.314 s 630.41 ms (7.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.298 s 614.504 ms (7.1%)
Total iast_TELEMETRY_OFF 9.31 s 626.1 ms (7.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent iast 1.186 s 125.146 ms (11.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.187 s 125.738 ms (11.8%)
Agent iast_TELEMETRY_OFF 1.182 s 120.543 ms (11.4%)
Total tracing 8.687 s -
Total iast 9.249 s 562.239 ms (6.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.259 s 572.123 ms (6.6%)
Total iast_TELEMETRY_OFF 9.282 s 595.007 ms (6.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~12273ce178, baseline=1.49.0-SNAPSHOT~b00c24e26a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.577 ms) : 0, 720577
BytebuddyAgent [candidate] (723.091 ms) : 0, 723091
GlobalTracer [baseline] (239.4 ms) : 0, 239400
GlobalTracer [candidate] (239.936 ms) : 0, 239936
AppSec [baseline] (55.705 ms) : 0, 55705
AppSec [candidate] (54.406 ms) : 0, 54406
Debugger [baseline] (5.057 ms) : 0, 5057
Debugger [candidate] (5.092 ms) : 0, 5092
Remote Config [baseline] (2.205 ms) : 0, 2205
Remote Config [candidate] (4.389 ms) : 0, 4389
Telemetry [baseline] (10.638 ms) : 0, 10638
Telemetry [candidate] (10.741 ms) : 0, 10741
section iast
BytebuddyAgent [baseline] (838.717 ms) : 0, 838717
BytebuddyAgent [candidate] (839.966 ms) : 0, 839966
GlobalTracer [baseline] (230.032 ms) : 0, 230032
GlobalTracer [candidate] (230.421 ms) : 0, 230421
AppSec [baseline] (55.09 ms) : 0, 55090
AppSec [candidate] (55.116 ms) : 0, 55116
Debugger [baseline] (4.209 ms) : 0, 4209
Debugger [candidate] (4.25 ms) : 0, 4250
Remote Config [baseline] (598.224 µs) : 0, 598
Remote Config [candidate] (616.008 µs) : 0, 616
Telemetry [baseline] (8.803 ms) : 0, 8803
Telemetry [candidate] (8.74 ms) : 0, 8740
IAST [baseline] (23.669 ms) : 0, 23669
IAST [candidate] (23.772 ms) : 0, 23772
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (841.363 ms) : 0, 841363
BytebuddyAgent [candidate] (840.66 ms) : 0, 840660
GlobalTracer [baseline] (230.21 ms) : 0, 230210
GlobalTracer [candidate] (230.1 ms) : 0, 230100
AppSec [baseline] (54.123 ms) : 0, 54123
AppSec [candidate] (54.605 ms) : 0, 54605
Debugger [baseline] (4.261 ms) : 0, 4261
Debugger [candidate] (4.237 ms) : 0, 4237
Remote Config [baseline] (616.59 µs) : 0, 617
Remote Config [candidate] (605.234 µs) : 0, 605
Telemetry [baseline] (8.766 ms) : 0, 8766
Telemetry [candidate] (8.73 ms) : 0, 8730
IAST [baseline] (24.48 ms) : 0, 24480
IAST [candidate] (24.434 ms) : 0, 24434
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (842.153 ms) : 0, 842153
BytebuddyAgent [candidate] (837.248 ms) : 0, 837248
GlobalTracer [baseline] (229.512 ms) : 0, 229512
GlobalTracer [candidate] (229.65 ms) : 0, 229650
AppSec [baseline] (55.631 ms) : 0, 55631
AppSec [candidate] (55.543 ms) : 0, 55543
Debugger [baseline] (4.142 ms) : 0, 4142
Debugger [candidate] (4.109 ms) : 0, 4109
Remote Config [baseline] (613.706 µs) : 0, 614
Remote Config [candidate] (620.582 µs) : 0, 621
Telemetry [baseline] (8.602 ms) : 0, 8602
Telemetry [candidate] (8.611 ms) : 0, 8611
IAST [baseline] (22.244 ms) : 0, 22244
IAST [candidate] (22.473 ms) : 0, 22473
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~12273ce178, baseline=1.49.0-SNAPSHOT~b00c24e26a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057484
Total [baseline] (10.462 s) : 0, 10461577
Agent [candidate] (1.057 s) : 0, 1056703
Total [candidate] (10.469 s) : 0, 10469400
section appsec
Agent [baseline] (1.2 s) : 0, 1200237
Total [baseline] (10.787 s) : 0, 10787326
Agent [candidate] (1.201 s) : 0, 1201097
Total [candidate] (10.815 s) : 0, 10815103
section iast
Agent [baseline] (1.196 s) : 0, 1195860
Total [baseline] (11.078 s) : 0, 11078178
Agent [candidate] (1.197 s) : 0, 1196861
Total [candidate] (11.108 s) : 0, 11108449
section profiling
Agent [baseline] (1.298 s) : 0, 1297872
Total [baseline] (10.999 s) : 0, 10999326
Agent [candidate] (1.295 s) : 0, 1295110
Total [candidate] (10.979 s) : 0, 10979372
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.2 s 142.753 ms (13.5%)
Agent iast 1.196 s 138.376 ms (13.1%)
Agent profiling 1.298 s 240.388 ms (22.7%)
Total tracing 10.462 s -
Total appsec 10.787 s 325.749 ms (3.1%)
Total iast 11.078 s 616.602 ms (5.9%)
Total profiling 10.999 s 537.749 ms (5.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.201 s 144.393 ms (13.7%)
Agent iast 1.197 s 140.157 ms (13.3%)
Agent profiling 1.295 s 238.406 ms (22.6%)
Total tracing 10.469 s -
Total appsec 10.815 s 345.703 ms (3.3%)
Total iast 11.108 s 639.049 ms (6.1%)
Total profiling 10.979 s 509.972 ms (4.9%) 
gantt
    title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~12273ce178, baseline=1.49.0-SNAPSHOT~b00c24e26a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (722.416 ms) : 0, 722416
BytebuddyAgent [candidate] (720.268 ms) : 0, 720268
GlobalTracer [baseline] (239.697 ms) : 0, 239697
GlobalTracer [candidate] (239.4 ms) : 0, 239400
AppSec [baseline] (55.111 ms) : 0, 55111
AppSec [candidate] (54.21 ms) : 0, 54210
Debugger [baseline] (4.422 ms) : 0, 4422
Debugger [candidate] (4.387 ms) : 0, 4387
Remote Config [baseline] (715.184 µs) : 0, 715
Remote Config [candidate] (693.652 µs) : 0, 694
Telemetry [baseline] (11.56 ms) : 0, 11560
Telemetry [candidate] (14.302 ms) : 0, 14302
section appsec
BytebuddyAgent [baseline] (740.371 ms) : 0, 740371
BytebuddyAgent [candidate] (740.917 ms) : 0, 740917
GlobalTracer [baseline] (236.801 ms) : 0, 236801
GlobalTracer [candidate] (236.444 ms) : 0, 236444
AppSec [baseline] (175.509 ms) : 0, 175509
AppSec [candidate] (175.581 ms) : 0, 175581
Debugger [baseline] (4.245 ms) : 0, 4245
Debugger [candidate] (5.005 ms) : 0, 5005
Remote Config [baseline] (643.426 µs) : 0, 643
Remote Config [candidate] (623.052 µs) : 0, 623
Telemetry [baseline] (8.232 ms) : 0, 8232
Telemetry [candidate] (8.171 ms) : 0, 8171
IAST [baseline] (21.964 ms) : 0, 21964
IAST [candidate] (21.812 ms) : 0, 21812
section iast
BytebuddyAgent [baseline] (847.423 ms) : 0, 847423
BytebuddyAgent [candidate] (848.534 ms) : 0, 848534
GlobalTracer [baseline] (232.271 ms) : 0, 232271
GlobalTracer [candidate] (231.888 ms) : 0, 231888
AppSec [baseline] (55.17 ms) : 0, 55170
AppSec [candidate] (55.386 ms) : 0, 55386
Debugger [baseline] (4.235 ms) : 0, 4235
Debugger [candidate] (4.29 ms) : 0, 4290
Remote Config [baseline] (607.772 µs) : 0, 608
Remote Config [candidate] (623.923 µs) : 0, 624
Telemetry [baseline] (8.776 ms) : 0, 8776
Telemetry [candidate] (8.794 ms) : 0, 8794
IAST [baseline] (23.84 ms) : 0, 23840
IAST [candidate] (23.793 ms) : 0, 23793
section profiling
BytebuddyAgent [baseline] (718.074 ms) : 0, 718074
BytebuddyAgent [candidate] (713.433 ms) : 0, 713433
GlobalTracer [baseline] (359.952 ms) : 0, 359952
GlobalTracer [candidate] (362.005 ms) : 0, 362005
AppSec [baseline] (54.118 ms) : 0, 54118
AppSec [candidate] (53.367 ms) : 0, 53367
Debugger [baseline] (4.422 ms) : 0, 4422
Debugger [candidate] (4.394 ms) : 0, 4394
Remote Config [baseline] (662.757 µs) : 0, 663
Remote Config [candidate] (653.202 µs) : 0, 653
Telemetry [baseline] (9.01 ms) : 0, 9010
Telemetry [candidate] (9.008 ms) : 0, 9008
ProfilingAgent [baseline] (101.03 ms) : 0, 101030
ProfilingAgent [candidate] (101.834 ms) : 0, 101834
Profiling [baseline] (101.055 ms) : 0, 101055
Profiling [candidate] (101.861 ms) : 0, 101861
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-04-16T17:10:17 2025-04-16T17:18:00
git_branch master landerson/remove-circleci-muzzle
git_commit_date 1744612276 1744821916
git_commit_sha b00c24e 12273ce
release_version 1.49.0-SNAPSHOT~b00c24e26a 1.49.0-SNAPSHOT~12273ce178
start_time 2025-04-16T17:10:03 2025-04-16T17:17:46
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1744824276 1744824276
ci_job_id 899981211 899981211
ci_pipeline_id 62397027 62397027
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner--31w5fpp-project-304-concurrent-0-y3we8iza 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner--31w5fpp-project-304-concurrent-0-y3we8iza 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 17 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~12273ce178, baseline=1.49.0-SNAPSHOT~b00c24e26a
    dateFormat X
    axisFormat %s
section baseline
no_agent (378.974 µs) : 358, 400
.   : milestone, 379,
iast (509.469 µs) : 488, 531
.   : milestone, 509,
iast_FULL (729.097 µs) : 707, 751
.   : milestone, 729,
iast_GLOBAL (554.656 µs) : 533, 576
.   : milestone, 555,
iast_HARDCODED_SECRET_DISABLED (509.637 µs) : 488, 531
.   : milestone, 510,
iast_INACTIVE (463.28 µs) : 442, 485
.   : milestone, 463,
iast_TELEMETRY_OFF (496.594 µs) : 475, 518
.   : milestone, 497,
tracing (457.835 µs) : 437, 479
.   : milestone, 458,
section candidate
no_agent (377.671 µs) : 358, 397
.   : milestone, 378,
iast (508.137 µs) : 486, 530
.   : milestone, 508,
iast_FULL (725.415 µs) : 703, 747
.   : milestone, 725,
iast_GLOBAL (558.323 µs) : 537, 580
.   : milestone, 558,
iast_HARDCODED_SECRET_DISABLED (510.489 µs) : 489, 532
.   : milestone, 510,
iast_INACTIVE (471.966 µs) : 449, 495
.   : milestone, 472,
iast_TELEMETRY_OFF (498.289 µs) : 476, 521
.   : milestone, 498,
tracing (454.734 µs) : 434, 476
.   : milestone, 455,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 378.974 µs [357.527 µs, 400.422 µs] -
iast 509.469 µs [487.917 µs, 531.021 µs] 130.495 µs (34.4%)
iast_FULL 729.097 µs [707.224 µs, 750.97 µs] 350.123 µs (92.4%)
iast_GLOBAL 554.656 µs [533.084 µs, 576.228 µs] 175.682 µs (46.4%)
iast_HARDCODED_SECRET_DISABLED 509.637 µs [487.802 µs, 531.473 µs] 130.663 µs (34.5%)
iast_INACTIVE 463.28 µs [442.017 µs, 484.543 µs] 84.306 µs (22.2%)
iast_TELEMETRY_OFF 496.594 µs [475.014 µs, 518.174 µs] 117.62 µs (31.0%)
tracing 457.835 µs [436.723 µs, 478.948 µs] 78.861 µs (20.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 377.671 µs [357.868 µs, 397.473 µs] -
iast 508.137 µs [486.436 µs, 529.839 µs] 130.467 µs (34.5%)
iast_FULL 725.415 µs [703.358 µs, 747.473 µs] 347.744 µs (92.1%)
iast_GLOBAL 558.323 µs [536.625 µs, 580.021 µs] 180.653 µs (47.8%)
iast_HARDCODED_SECRET_DISABLED 510.489 µs [488.758 µs, 532.22 µs] 132.818 µs (35.2%)
iast_INACTIVE 471.966 µs [449.425 µs, 494.508 µs] 94.296 µs (25.0%)
iast_TELEMETRY_OFF 498.289 µs [476.075 µs, 520.504 µs] 120.619 µs (31.9%)
tracing 454.734 µs [433.797 µs, 475.67 µs] 77.063 µs (20.4%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~12273ce178, baseline=1.49.0-SNAPSHOT~b00c24e26a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.356 ms) : 1336, 1375
.   : milestone, 1356,
appsec (1.743 ms) : 1718, 1767
.   : milestone, 1743,
appsec_no_iast (1.739 ms) : 1715, 1763
.   : milestone, 1739,
code_origins (1.676 ms) : 1648, 1703
.   : milestone, 1676,
iast (1.498 ms) : 1474, 1523
.   : milestone, 1498,
profiling (1.52 ms) : 1496, 1545
.   : milestone, 1520,
tracing (1.49 ms) : 1466, 1515
.   : milestone, 1490,
section candidate
no_agent (1.344 ms) : 1324, 1364
.   : milestone, 1344,
appsec (1.745 ms) : 1721, 1769
.   : milestone, 1745,
appsec_no_iast (1.731 ms) : 1707, 1755
.   : milestone, 1731,
code_origins (1.687 ms) : 1660, 1714
.   : milestone, 1687,
iast (1.504 ms) : 1480, 1529
.   : milestone, 1504,
profiling (1.485 ms) : 1462, 1509
.   : milestone, 1485,
tracing (1.491 ms) : 1467, 1516
.   : milestone, 1491,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.356 ms [1.336 ms, 1.375 ms] -
appsec 1.743 ms [1.718 ms, 1.767 ms] 387.234 µs (28.6%)
appsec_no_iast 1.739 ms [1.715 ms, 1.763 ms] 383.037 µs (28.3%)
code_origins 1.676 ms [1.648 ms, 1.703 ms] 320.221 µs (23.6%)
iast 1.498 ms [1.474 ms, 1.523 ms] 142.803 µs (10.5%)
profiling 1.52 ms [1.496 ms, 1.545 ms] 164.782 µs (12.2%)
tracing 1.49 ms [1.466 ms, 1.515 ms] 134.744 µs (9.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.344 ms [1.324 ms, 1.364 ms] -
appsec 1.745 ms [1.721 ms, 1.769 ms] 401.107 µs (29.8%)
appsec_no_iast 1.731 ms [1.707 ms, 1.755 ms] 387.147 µs (28.8%)
code_origins 1.687 ms [1.66 ms, 1.714 ms] 342.781 µs (25.5%)
iast 1.504 ms [1.48 ms, 1.529 ms] 160.439 µs (11.9%)
profiling 1.485 ms [1.462 ms, 1.509 ms] 141.527 µs (10.5%)
tracing 1.491 ms [1.467 ms, 1.516 ms] 147.471 µs (11.0%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master landerson/remove-circleci-muzzle
git_commit_date 1744612276 1744821916
git_commit_sha b00c24e 12273ce
release_version 1.49.0-SNAPSHOT~b00c24e26a 1.49.0-SNAPSHOT~12273ce178
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1744824791 1744824791
ci_job_id 899981213 899981213
ci_pipeline_id 62397027 62397027
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-tymfbhrg-project-304-concurrent-1-1y8in60m 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-tymfbhrg-project-304-concurrent-1-1y8in60m 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~12273ce178, baseline=1.49.0-SNAPSHOT~b00c24e26a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.477 ms) : 1466, 1489
.   : milestone, 1477,
appsec (2.351 ms) : 2308, 2395
.   : milestone, 2351,
iast (2.125 ms) : 2070, 2180
.   : milestone, 2125,
iast_GLOBAL (2.17 ms) : 2114, 2226
.   : milestone, 2170,
profiling (1.98 ms) : 1937, 2023
.   : milestone, 1980,
tracing (1.957 ms) : 1915, 1999
.   : milestone, 1957,
section candidate
no_agent (1.478 ms) : 1466, 1489
.   : milestone, 1478,
appsec (2.361 ms) : 2317, 2405
.   : milestone, 2361,
iast (2.124 ms) : 2069, 2180
.   : milestone, 2124,
iast_GLOBAL (2.172 ms) : 2116, 2228
.   : milestone, 2172,
profiling (2.433 ms) : 2255, 2611
.   : milestone, 2433,
tracing (1.963 ms) : 1920, 2005
.   : milestone, 1963,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.477 ms [1.466 ms, 1.489 ms] -
appsec 2.351 ms [2.308 ms, 2.395 ms] 874.109 µs (59.2%)
iast 2.125 ms [2.07 ms, 2.18 ms] 648.039 µs (43.9%)
iast_GLOBAL 2.17 ms [2.114 ms, 2.226 ms] 692.963 µs (46.9%)
profiling 1.98 ms [1.937 ms, 2.023 ms] 502.936 µs (34.0%)
tracing 1.957 ms [1.915 ms, 1.999 ms] 479.949 µs (32.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.478 ms [1.466 ms, 1.489 ms] -
appsec 2.361 ms [2.317 ms, 2.405 ms] 883.045 µs (59.8%)
iast 2.124 ms [2.069 ms, 2.18 ms] 646.413 µs (43.7%)
iast_GLOBAL 2.172 ms [2.116 ms, 2.228 ms] 694.01 µs (47.0%)
profiling 2.433 ms [2.255 ms, 2.611 ms] 955.341 µs (64.6%)
tracing 1.963 ms [1.92 ms, 2.005 ms] 484.85 µs (32.8%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~12273ce178, baseline=1.49.0-SNAPSHOT~b00c24e26a
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.879 s) : 14879000, 14879000
.   : milestone, 14879000,
appsec (15.077 s) : 15077000, 15077000
.   : milestone, 15077000,
iast (18.866 s) : 18866000, 18866000
.   : milestone, 18866000,
iast_GLOBAL (17.872 s) : 17872000, 17872000
.   : milestone, 17872000,
profiling (14.958 s) : 14958000, 14958000
.   : milestone, 14958000,
tracing (15.083 s) : 15083000, 15083000
.   : milestone, 15083000,
section candidate
no_agent (15.579 s) : 15579000, 15579000
.   : milestone, 15579000,
appsec (14.789 s) : 14789000, 14789000
.   : milestone, 14789000,
iast (18.637 s) : 18637000, 18637000
.   : milestone, 18637000,
iast_GLOBAL (17.867 s) : 17867000, 17867000
.   : milestone, 17867000,
profiling (15.549 s) : 15549000, 15549000
.   : milestone, 15549000,
tracing (14.921 s) : 14921000, 14921000
.   : milestone, 14921000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.879 s [14.879 s, 14.879 s] -
appsec 15.077 s [15.077 s, 15.077 s] 198.0 ms (1.3%)
iast 18.866 s [18.866 s, 18.866 s] 3.987 s (26.8%)
iast_GLOBAL 17.872 s [17.872 s, 17.872 s] 2.993 s (20.1%)
profiling 14.958 s [14.958 s, 14.958 s] 79.0 ms (0.5%)
tracing 15.083 s [15.083 s, 15.083 s] 204.0 ms (1.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.579 s [15.579 s, 15.579 s] -
appsec 14.789 s [14.789 s, 14.789 s] -790.0 ms (-5.1%)
iast 18.637 s [18.637 s, 18.637 s] 3.058 s (19.6%)
iast_GLOBAL 17.867 s [17.867 s, 17.867 s] 2.288 s (14.7%)
profiling 15.549 s [15.549 s, 15.549 s] -30.0 ms (-0.2%)
tracing 14.921 s [14.921 s, 14.921 s] -658.0 ms (-4.2%)

smola
smola reviewed Apr 21, 2025
View reviewed changes
Copy link
Member

@smola smola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a required job to the GitLab pipeline and set it to required for merge in the GitHub settings.

PerfectSlayer
PerfectSlayer reviewed Apr 22, 2025
View reviewed changes
Copy link
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would muzzle runs consistently in GitLab without a pro Nexus? (just checking)

@randomanderson
Copy link
Contributor Author

@PerfectSlayer Pro Nexus is more of a nice to have than a necessity. muzzle has been running on all branches for ~2 months now with the major proxy issues ironed out ~1 month ago.

@randomanderson
Copy link
Contributor Author

@smola the required job in gitlab already exists. When all tests are enabled, it might not be possible because of Gitlab's dependency limit. At that point, we can require the entire pipeline passing

@PerfectSlayer
Copy link
Contributor

So is muzzle running from GitLab a required GitHub PR check already?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smola smola smola left review comments

@PerfectSlayer PerfectSlayer PerfectSlayer left review comments

@ygree ygree Awaiting requested review from ygree ygree is a code owner automatically assigned from DataDog/apm-java

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@randomanderson @PerfectSlayer @smola