Skip to content

Update AgentTestRunner to use JUnit5 - Continuation #9413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 56 commits into
base: nikita-tkachenko/junit-6
Choose a base branch
from

Conversation

sarahchen6
Copy link
Contributor

What Does This Do

Built on #9264 to address failing instrumentation tests

Motivation

Update AgentTestRunner to use JUnit5, necessary to support Java 25 and test JUnit6

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

amarziali and others added 30 commits August 8, 2025 13:59
* Support async servlet for RUM injection

* Move isRumEnabled to instrumenter config
* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* Rollback some dependencies to fix compilation.

* Added dependency of `compilePlayRoutes` on `fixPlayRoutesDirectory`.

* Rollback some dependencies to fix compilation.

---------

Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>
Co-authored-by: Alexey Kuznetsov <[email protected]>
Bumps the gh-actions-packages group with 3 updates: [actions/checkout](https://github.com/actions/checkout), [actions/cache](https://github.com/actions/cache) and [github/codeql-action](https://github.com/github/codeql-action).


Updates `actions/checkout` from 4.2.2 to 5.0.0
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@11bd719...08c6903)

Updates `actions/cache` from 4.2.3 to 4.2.4
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](actions/cache@5a3ec84...0400d5f)

Updates `github/codeql-action` from 3.29.7 to 3.29.8
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@51f7732...76621b6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 5.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: gh-actions-packages
- dependency-name: actions/cache
  dependency-version: 4.2.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: gh-actions-packages
- dependency-name: github/codeql-action
  dependency-version: 3.29.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: gh-actions-packages
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Avoid NPE on featureDiscovery creation

* Review
What Does This Do
Modifies WafModule#buildEvents to safely handle actionWithData.data being null.

Motivation
Fix #9346

Additional Notes
The current version of libddwaf may return null in the data field of actionWithData. This was previously not handled and could cause unexpected logged exceptions. The method now explicitly checks for null to prevent this.
#9351)

* use magic mirror depot

* remove separate cache name used for testing
by eliminating the chance identity hash might collide even after 3 rehashes
the easiest way to do this is to bump the size of the cache used in testing

note this is only necessary because we want to keep both keys in the cache
to test the 'visit' method - otherwise we'd need to relax that assertion
* Add dependency to OCI publish jobs

* Override new publishing-gate job

* Change stage order because publishing-gate depends on deploy_to_maven_central

* Add verify_maven_central_deployment job

* Test verify_maven_central_deployment job by temporarily removing dependency

* Change to registry.ddbuild image

* Test failing version

* Clean

* Change image for verify_maven_central_deployment job

* Fix verify_maven_central_deployment script

* Remove publishing-gate rules

* Test verify job

* Add manual trigger

* Fix verify script

* Add test-job to see what happens when verify job fails

* Clean test code

* Change retry rate

* Try test-job again

* Clean again

* Create a separate job to skip the verify maven job

---------

Co-authored-by: Laplie Anderson <[email protected]>
* feat(env): Migrate internal apis to environment component
* feat(env): Add new API to EnvironmentVariables and SystemProperties
* Refactor matchOperator to use method reference comparator for efficiency

* Implement better exception handling and debug logging for StableConfig parsing errors

* Add a debug log about no rules getting applied

* Improve test file logic and cleanup in StableConfigSourceTest class

* Github suggestions: Use static factory methods  as constructors for Rule and Selector classes

* Improve exception message: include value of object that caused the failure

* Add more test cases to StableConfigSourceTest to improve codecov

* Add test coverage for StableConfigMappingException

* Update internal-api/src/main/java/datadog/trace/bootstrap/config/provider/stableconfig/Rule.java

Co-authored-by: Alexey Kuznetsov <[email protected]>

* Update internal-api/src/main/java/datadog/trace/bootstrap/config/provider/stableconfig/Rule.java

Co-authored-by: Alexey Kuznetsov <[email protected]>

* Use safeToString when printing stableconfig map

* rearrange logic in selectors creation for better readability

* delete superfluous else

* Use constant to represent max length in safeToString

* use throwStableConfigMappingException helper function

* refactor exception catching in StableConfigSource

* restore semicolon in calls to throwStableConfigMappingException

* Use static import for throwStableConfigMappingException

* test other kinds of exceptions in StableConfigSource

* remove duplicate cfg assignments

* fix missing operator test case

* Update internal-api/src/main/java/datadog/trace/bootstrap/config/provider/stableconfig/Rule.java

Co-authored-by: Alexey Kuznetsov <[email protected]>

* Improve readability of StableConfigMappingException and its function invocations

* modify safeToString logic to print first 50 and last 50 chars

* remove extra placeholder in string message when printing exception

* Update internal-api/src/main/java/datadog/trace/bootstrap/config/provider/StableConfigSource.java

Co-authored-by: Alexey Kuznetsov <[email protected]>

* refactor Rules.from to perform type assertions before heavyweight list manipulation

* run spotless

---------

Co-authored-by: Alexey Kuznetsov <[email protected]>
* Add stage for publishing-gate

* Allow publishing-gate to run when populating cache
* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* chore: Update Gradle dependencies

* Rollback some dependencies to fix compilation.

---------

Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>
Co-authored-by: Alexey Kuznetsov <[email protected]>
* feat(bootstrap): Migrate Spray instrumentation to Context API
* feat(bootstrap): Migrate Tomcat instrumentation to Context API
* feat(bootstrap): Migrate Liberty instrumentations to Context API
* feat(bootstrap): Migrate instrumentations to Context API
* fix(bootstrap): Fix unexpected call to IG callback
* feat(play): Simplify testing
* fix(bootstrap): Fix import
Do not stop on git log check
Fix next patch release version compute
…generated keys (#9321)

* Prevent crash in SQL Server JDBC when tracing execute methods with generated keys

When tracing the JDBC statement execute methods, we prepend a comment to
the SQL query used for DBM trace propagation.
However, there is a bug in the SQL Server JDBC driver that prevents the
generated keys from being returned when the SQL comment is prepended to
the SQL query.
I decided to only append the comment in this specific case to avoid the
comment from being truncated.

@see microsoft/mssql-jdbc#2729

* remove dynamic typing when packing all the JDBC arguments

* move JDBC test on injection with generated keys to SQL Server file

* remove useless constant in StatementAdvice
…#9382)

Bumps the gh-actions-packages group with 1 update: [github/codeql-action](https://github.com/github/codeql-action).


Updates `github/codeql-action` from 3.29.8 to 3.29.10
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@76621b6...96f518a)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 3.29.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: gh-actions-packages
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat(dsm): Improve code style
* feat(aws): Refactor AWS instrumentation around Context
amarziali and others added 26 commits August 19, 2025 09:56
* Implement health metrics for client stats

* Use same naming than dd-trace-go

* format
* Add details to the README about GitHub actions tests
---------

Co-authored-by: Bruce Bujon <[email protected]>
…s.code` (#9367)

* Changing GRPC instrumentation codes from status.code to grpc.status.code

* adding grpc.status.code to pubsub tests

* updating to have both tags be sent
* Code clean up

* Ensure minimum number of fuzz tests on TagsMap

* suggestion
* Add RumInjectorHealthMetrics

* Add telemetry collector and methods to RumInjector

* Initialize health metrics and telemetry collector

* Get injectionsucceed count

* Add comments

* Reorganize classes

* Connect rum injector, telemetry collector, and statsdclient

* Add tests

* Get and test metrics for injection failures and skips

* Add Content Security Policy and HTTP response size telemetry

* Add injection duration telemetry

* Fix some things

* Fix content-length retrieval and add test for injection timing

* Add injection initialization success telemetry

* Fix CoreTracer compilation with InstrumenterConfig

* Add tags to all metrics

* Update InjectingPipeOutputStreamTest

* Tweaks

* Address jacoco coverage and injectingpipeoutstream interface updates

* Add content-length detection for InjectingPipeWriter and improve tests

* Address review comments

* Fix header retrieval

* Add lots of improvements from review comments

* Fix constructors and address review comment

* Clarify bytes written and address review comment

* Use dynamic servlet version retrieval

* Change injection timing logic

* Clean up

* Use dynamic servlet version retrieval for tagging as well

* Add a telemetry check to HttpServerTest

* Clean up
* Use origin CALCULATED for merged methods

* fix javadoc for ConfigOrigin.CALCULATED

* apply github suggestions
…9370)

* throw IllegalArgumentException in ConfigConverter.booleanValueOf

* modify get to only catch IllegalArgumentException

* Fix failing ConfigTest tests

* Update dd-java-agent/agent-bootstrap/src/main/java/datadog/trace/bootstrap/instrumentation/decorator/BaseDecorator.java

Co-authored-by: Stuart McCulloch <[email protected]>

* Use InvalidBooleanValueException; return false when encountered

---------

Co-authored-by: Stuart McCulloch <[email protected]>
* Add ErrorPriorities for manual instrumentation and tag interceptor

(cherry picked from commit c26ef68)

* Set resource name and error priority to manual instrumentation for otel spans

(cherry picked from commit f573ade)

* remove tag interceptor error priority

* Test on new error priorities level

---------

Co-authored-by: DJ Gregor <[email protected]>
…current thread is not the original. (#9403)

This mitigates a potential issue if the swapped out scope-stack is ever restored more than once on different threads.
* initial introduction of config_id

* fix tests, fix NPE in ConfigSetting.equals, include configId in telemetryrequestbody when nonnull

* report config id for stableconfigsources in get

* Fix call to outdated ConfigSetting constructor in EventSourceTest

* fix config_id report

* fix where null config id was getting treated as string 'null'

* report config id for all get methods

* remove superfluous file

* simplify configId resolution logic in configProvider

* report sourceValue instead of value to ConfigCollector in get method

* add tests for configid and no configid
Copy link

datadog-official bot commented Aug 22, 2025

🎯 Code Coverage
Patch Coverage: 82.59%
Total Coverage: 59.87% (+0.27%)

View detailed report

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9871ee0 | Docs | Was this helpful? Give us feedback!

@pr-commenter
Copy link

pr-commenter bot commented Aug 22, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/experiment
git_commit_date 1755864261 1755891364
git_commit_sha 0ce85a2 9871ee0
release_version 1.53.0-SNAPSHOT~0ce85a2743 1.53.0-SNAPSHOT~9871ee0307
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1755893215 1755893215
ci_job_id 1095096693 1095096693
ci_pipeline_id 74529941 74529941
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-7wrss4z3 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-7wrss4z3 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 11 unstable metrics.

Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~9871ee0307, baseline=1.53.0-SNAPSHOT~0ce85a2743

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.053 s) : 0, 1053144
Total [baseline] (8.62 s) : 0, 8620121
Agent [candidate] (1.047 s) : 0, 1046573
Total [candidate] (8.612 s) : 0, 8611549
section iast
Agent [baseline] (1.178 s) : 0, 1177947
Total [baseline] (9.31 s) : 0, 9310290
Agent [candidate] (1.179 s) : 0, 1178949
Total [candidate] (9.36 s) : 0, 9359832
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.053 s -
Agent iast 1.178 s 124.803 ms (11.9%)
Total tracing 8.62 s -
Total iast 9.31 s 690.169 ms (8.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.047 s -
Agent iast 1.179 s 132.376 ms (12.6%)
Total tracing 8.612 s -
Total iast 9.36 s 748.282 ms (8.7%)
gantt
    title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~9871ee0307, baseline=1.53.0-SNAPSHOT~0ce85a2743

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.463 ms) : 0, 1463
crashtracking [candidate] (1.453 ms) : 0, 1453
BytebuddyAgent [baseline] (736.187 ms) : 0, 736187
BytebuddyAgent [candidate] (733.686 ms) : 0, 733686
GlobalTracer [baseline] (243.897 ms) : 0, 243897
GlobalTracer [candidate] (242.795 ms) : 0, 242795
AppSec [baseline] (30.487 ms) : 0, 30487
AppSec [candidate] (30.176 ms) : 0, 30176
Debugger [baseline] (6.137 ms) : 0, 6137
Debugger [candidate] (6.08 ms) : 0, 6080
Remote Config [baseline] (685.862 µs) : 0, 686
Remote Config [candidate] (669.951 µs) : 0, 670
Telemetry [baseline] (13.188 ms) : 0, 13188
Telemetry [candidate] (10.753 ms) : 0, 10753
section iast
crashtracking [baseline] (1.456 ms) : 0, 1456
crashtracking [candidate] (1.454 ms) : 0, 1454
BytebuddyAgent [baseline] (850.785 ms) : 0, 850785
BytebuddyAgent [candidate] (850.586 ms) : 0, 850586
GlobalTracer [baseline] (232.424 ms) : 0, 232424
GlobalTracer [candidate] (233.312 ms) : 0, 233312
IAST [baseline] (30.087 ms) : 0, 30087
IAST [candidate] (28.561 ms) : 0, 28561
AppSec [baseline] (26.05 ms) : 0, 26050
AppSec [candidate] (26.779 ms) : 0, 26779
Debugger [baseline] (7.418 ms) : 0, 7418
Debugger [candidate] (8.337 ms) : 0, 8337
Remote Config [baseline] (602.284 µs) : 0, 602
Remote Config [candidate] (601.234 µs) : 0, 601
Telemetry [baseline] (8.211 ms) : 0, 8211
Telemetry [candidate] (8.279 ms) : 0, 8279
Loading
Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~9871ee0307, baseline=1.53.0-SNAPSHOT~0ce85a2743

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.054 s) : 0, 1054355
Total [baseline] (10.726 s) : 0, 10726420
Agent [candidate] (1.056 s) : 0, 1055633
Total [candidate] (10.712 s) : 0, 10712080
section appsec
Agent [baseline] (1.232 s) : 0, 1232006
Total [baseline] (10.909 s) : 0, 10909201
Agent [candidate] (1.225 s) : 0, 1224719
Total [candidate] (10.862 s) : 0, 10861725
section iast
Agent [baseline] (1.181 s) : 0, 1180541
Total [baseline] (10.934 s) : 0, 10934036
Agent [candidate] (1.181 s) : 0, 1180649
Total [candidate] (10.983 s) : 0, 10983406
section profiling
Agent [baseline] (1.199 s) : 0, 1198972
Total [baseline] (10.922 s) : 0, 10922338
Agent [candidate] (1.202 s) : 0, 1201970
Total [candidate] (10.892 s) : 0, 10892044
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent appsec 1.232 s 177.651 ms (16.8%)
Agent iast 1.181 s 126.186 ms (12.0%)
Agent profiling 1.199 s 144.617 ms (13.7%)
Total tracing 10.726 s -
Total appsec 10.909 s 182.781 ms (1.7%)
Total iast 10.934 s 207.615 ms (1.9%)
Total profiling 10.922 s 195.918 ms (1.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.056 s -
Agent appsec 1.225 s 169.086 ms (16.0%)
Agent iast 1.181 s 125.016 ms (11.8%)
Agent profiling 1.202 s 146.337 ms (13.9%)
Total tracing 10.712 s -
Total appsec 10.862 s 149.645 ms (1.4%)
Total iast 10.983 s 271.327 ms (2.5%)
Total profiling 10.892 s 179.965 ms (1.7%)
gantt
    title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~9871ee0307, baseline=1.53.0-SNAPSHOT~0ce85a2743

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.468 ms) : 0, 1468
crashtracking [candidate] (1.449 ms) : 0, 1449
BytebuddyAgent [baseline] (737.839 ms) : 0, 737839
BytebuddyAgent [candidate] (737.793 ms) : 0, 737793
GlobalTracer [baseline] (244.11 ms) : 0, 244110
GlobalTracer [candidate] (244.42 ms) : 0, 244420
AppSec [baseline] (30.467 ms) : 0, 30467
AppSec [candidate] (30.322 ms) : 0, 30322
Debugger [baseline] (6.087 ms) : 0, 6087
Debugger [candidate] (6.109 ms) : 0, 6109
Remote Config [baseline] (677.374 µs) : 0, 677
Remote Config [candidate] (662.758 µs) : 0, 663
Telemetry [baseline] (12.445 ms) : 0, 12445
Telemetry [candidate] (13.75 ms) : 0, 13750
section appsec
crashtracking [baseline] (1.459 ms) : 0, 1459
crashtracking [candidate] (1.452 ms) : 0, 1452
BytebuddyAgent [baseline] (761.07 ms) : 0, 761070
BytebuddyAgent [candidate] (755.947 ms) : 0, 755947
GlobalTracer [baseline] (236.937 ms) : 0, 236937
GlobalTracer [candidate] (235.372 ms) : 0, 235372
IAST [baseline] (23.622 ms) : 0, 23622
IAST [candidate] (23.514 ms) : 0, 23514
AppSec [baseline] (171.067 ms) : 0, 171067
AppSec [candidate] (169.958 ms) : 0, 169958
Debugger [baseline] (5.862 ms) : 0, 5862
Debugger [candidate] (7.357 ms) : 0, 7357
Remote Config [baseline] (646.261 µs) : 0, 646
Remote Config [candidate] (652.547 µs) : 0, 653
Telemetry [baseline] (10.044 ms) : 0, 10044
Telemetry [candidate] (9.269 ms) : 0, 9269
section iast
crashtracking [baseline] (1.46 ms) : 0, 1460
crashtracking [candidate] (1.446 ms) : 0, 1446
BytebuddyAgent [baseline] (852.022 ms) : 0, 852022
BytebuddyAgent [candidate] (851.489 ms) : 0, 851489
GlobalTracer [baseline] (233.324 ms) : 0, 233324
GlobalTracer [candidate] (233.391 ms) : 0, 233391
IAST [baseline] (31.222 ms) : 0, 31222
IAST [candidate] (28.487 ms) : 0, 28487
AppSec [baseline] (26.731 ms) : 0, 26731
AppSec [candidate] (29.869 ms) : 0, 29869
Debugger [baseline] (5.762 ms) : 0, 5762
Debugger [candidate] (5.795 ms) : 0, 5795
Remote Config [baseline] (610.014 µs) : 0, 610
Remote Config [candidate] (615.197 µs) : 0, 615
Telemetry [baseline] (8.378 ms) : 0, 8378
Telemetry [candidate] (8.479 ms) : 0, 8479
section profiling
crashtracking [baseline] (1.443 ms) : 0, 1443
crashtracking [candidate] (1.452 ms) : 0, 1452
BytebuddyAgent [baseline] (763.019 ms) : 0, 763019
BytebuddyAgent [candidate] (765.859 ms) : 0, 765859
GlobalTracer [baseline] (222.376 ms) : 0, 222376
GlobalTracer [candidate] (222.944 ms) : 0, 222944
AppSec [baseline] (30.965 ms) : 0, 30965
AppSec [candidate] (30.893 ms) : 0, 30893
Debugger [baseline] (6.366 ms) : 0, 6366
Debugger [candidate] (6.313 ms) : 0, 6313
Remote Config [baseline] (713.242 µs) : 0, 713
Remote Config [candidate] (701.969 µs) : 0, 702
Telemetry [baseline] (15.153 ms) : 0, 15153
Telemetry [candidate] (14.859 ms) : 0, 14859
ProfilingAgent [baseline] (109.116 ms) : 0, 109116
ProfilingAgent [candidate] (108.241 ms) : 0, 108241
Profiling [baseline] (109.752 ms) : 0, 109752
Profiling [candidate] (108.884 ms) : 0, 108884
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/experiment
git_commit_date 1755864261 1755891364
git_commit_sha 0ce85a2 9871ee0
release_version 1.53.0-SNAPSHOT~0ce85a2743 1.53.0-SNAPSHOT~9871ee0307
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1755892889 1755892889
ci_job_id 1095096694 1095096694
ci_pipeline_id 74529941 74529941
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-h99evp9m 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-h99evp9m 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 2 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:no_agent:high_load worse
[+103.475µs; +212.178µs] or [+2.370%; +4.860%]
unstable
[-139.302op/s; +131.024op/s] or [-13.663%; +12.851%]
4.524ms 1015.406op/s 4.366ms 1019.545op/s
scenario:load:insecure-bank:tracing:high_load worse
[+330.533µs; +568.048µs] or [+4.493%; +7.722%]
unstable
[-116.188op/s; +44.375op/s] or [-18.462%; +7.051%]
7.806ms 593.438op/s 7.356ms 629.344op/s
scenario:load:insecure-bank:iast:high_load better
[-768.970µs; -432.653µs] or [-7.845%; -4.414%]
unstable
[-29.098op/s; +90.035op/s] or [-6.141%; +19.002%]
9.201ms 504.281op/s 9.802ms 473.812op/s
scenario:load:petclinic:iast:high_load better
[-3.131ms; -2.309ms] or [-6.765%; -4.988%]
unstable
[-0.773op/s; +13.223op/s] or [-0.764%; +13.073%]
43.570ms 107.375op/s 46.290ms 101.150op/s
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~9871ee0307, baseline=1.53.0-SNAPSHOT~0ce85a2743
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.366 ms) : 4316, 4416
.   : milestone, 4366,
iast (9.802 ms) : 9641, 9963
.   : milestone, 9802,
iast_FULL (14.07 ms) : 13790, 14349
.   : milestone, 14070,
iast_GLOBAL (10.238 ms) : 10060, 10415
.   : milestone, 10238,
profiling (8.927 ms) : 8787, 9067
.   : milestone, 8927,
tracing (7.356 ms) : 7247, 7466
.   : milestone, 7356,
section candidate
no_agent (4.524 ms) : 4473, 4575
.   : milestone, 4524,
iast (9.201 ms) : 9050, 9353
.   : milestone, 9201,
iast_FULL (13.779 ms) : 13515, 14043
.   : milestone, 13779,
iast_GLOBAL (10.088 ms) : 9905, 10271
.   : milestone, 10088,
profiling (8.734 ms) : 8593, 8874
.   : milestone, 8734,
tracing (7.806 ms) : 7694, 7917
.   : milestone, 7806,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.366 ms [4.316 ms, 4.416 ms] -
iast 9.802 ms [9.641 ms, 9.963 ms] 5.436 ms (124.5%)
iast_FULL 14.07 ms [13.79 ms, 14.349 ms] 9.704 ms (222.3%)
iast_GLOBAL 10.238 ms [10.06 ms, 10.415 ms] 5.872 ms (134.5%)
profiling 8.927 ms [8.787 ms, 9.067 ms] 4.561 ms (104.5%)
tracing 7.356 ms [7.247 ms, 7.466 ms] 2.99 ms (68.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.524 ms [4.473 ms, 4.575 ms] -
iast 9.201 ms [9.05 ms, 9.353 ms] 4.678 ms (103.4%)
iast_FULL 13.779 ms [13.515 ms, 14.043 ms] 9.255 ms (204.6%)
iast_GLOBAL 10.088 ms [9.905 ms, 10.271 ms] 5.564 ms (123.0%)
profiling 8.734 ms [8.593 ms, 8.874 ms] 4.21 ms (93.1%)
tracing 7.806 ms [7.694 ms, 7.917 ms] 3.282 ms (72.5%)
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~9871ee0307, baseline=1.53.0-SNAPSHOT~0ce85a2743
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.59 ms) : 37279, 37902
.   : milestone, 37590,
appsec (47.587 ms) : 47140, 48033
.   : milestone, 47587,
code_origins (44.81 ms) : 44432, 45188
.   : milestone, 44810,
iast (46.29 ms) : 45886, 46695
.   : milestone, 46290,
profiling (47.229 ms) : 46826, 47632
.   : milestone, 47229,
tracing (44.007 ms) : 43634, 44381
.   : milestone, 44007,
section candidate
no_agent (37.167 ms) : 36868, 37466
.   : milestone, 37167,
appsec (48.761 ms) : 48309, 49214
.   : milestone, 48761,
code_origins (43.551 ms) : 43199, 43903
.   : milestone, 43551,
iast (43.57 ms) : 43212, 43928
.   : milestone, 43570,
profiling (47.544 ms) : 47080, 48008
.   : milestone, 47544,
tracing (45.22 ms) : 44835, 45606
.   : milestone, 45220,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 37.59 ms [37.279 ms, 37.902 ms] -
appsec 47.587 ms [47.14 ms, 48.033 ms] 9.996 ms (26.6%)
code_origins 44.81 ms [44.432 ms, 45.188 ms] 7.22 ms (19.2%)
iast 46.29 ms [45.886 ms, 46.695 ms] 8.7 ms (23.1%)
profiling 47.229 ms [46.826 ms, 47.632 ms] 9.638 ms (25.6%)
tracing 44.007 ms [43.634 ms, 44.381 ms] 6.417 ms (17.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 37.167 ms [36.868 ms, 37.466 ms] -
appsec 48.761 ms [48.309 ms, 49.214 ms] 11.595 ms (31.2%)
code_origins 43.551 ms [43.199 ms, 43.903 ms] 6.384 ms (17.2%)
iast 43.57 ms [43.212 ms, 43.928 ms] 6.403 ms (17.2%)
profiling 47.544 ms [47.08 ms, 48.008 ms] 10.377 ms (27.9%)
tracing 45.22 ms [44.835 ms, 45.606 ms] 8.054 ms (21.7%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/experiment
git_commit_date 1755864261 1755891364
git_commit_sha 0ce85a2 9871ee0
release_version 1.53.0-SNAPSHOT~0ce85a2743 1.53.0-SNAPSHOT~9871ee0307
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1755893367 1755893367
ci_job_id 1095096695 1095096695
ci_pipeline_id 74529941 74529941
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-0afklh27 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-0afklh27 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~9871ee0307, baseline=1.53.0-SNAPSHOT~0ce85a2743
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.478 ms) : 1466, 1489
.   : milestone, 1478,
appsec (3.673 ms) : 3456, 3891
.   : milestone, 3673,
iast (2.203 ms) : 2140, 2266
.   : milestone, 2203,
iast_GLOBAL (2.245 ms) : 2181, 2308
.   : milestone, 2245,
profiling (2.494 ms) : 2324, 2663
.   : milestone, 2494,
tracing (2.013 ms) : 1965, 2062
.   : milestone, 2013,
section candidate
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (3.654 ms) : 3438, 3870
.   : milestone, 3654,
iast (2.199 ms) : 2137, 2262
.   : milestone, 2199,
iast_GLOBAL (2.246 ms) : 2183, 2309
.   : milestone, 2246,
profiling (2.059 ms) : 2007, 2110
.   : milestone, 2059,
tracing (2.007 ms) : 1958, 2056
.   : milestone, 2007,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.478 ms [1.466 ms, 1.489 ms] -
appsec 3.673 ms [3.456 ms, 3.891 ms] 2.196 ms (148.6%)
iast 2.203 ms [2.14 ms, 2.266 ms] 725.029 µs (49.1%)
iast_GLOBAL 2.245 ms [2.181 ms, 2.308 ms] 766.743 µs (51.9%)
profiling 2.494 ms [2.324 ms, 2.663 ms] 1.016 ms (68.7%)
tracing 2.013 ms [1.965 ms, 2.062 ms] 535.617 µs (36.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.463 ms, 1.486 ms] -
appsec 3.654 ms [3.438 ms, 3.87 ms] 2.179 ms (147.8%)
iast 2.199 ms [2.137 ms, 2.262 ms] 724.599 µs (49.1%)
iast_GLOBAL 2.246 ms [2.183 ms, 2.309 ms] 771.19 µs (52.3%)
profiling 2.059 ms [2.007 ms, 2.11 ms] 583.825 µs (39.6%)
tracing 2.007 ms [1.958 ms, 2.056 ms] 532.518 µs (36.1%)
Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~9871ee0307, baseline=1.53.0-SNAPSHOT~0ce85a2743
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.723 s) : 15723000, 15723000
.   : milestone, 15723000,
appsec (14.974 s) : 14974000, 14974000
.   : milestone, 14974000,
iast (18.343 s) : 18343000, 18343000
.   : milestone, 18343000,
iast_GLOBAL (18.234 s) : 18234000, 18234000
.   : milestone, 18234000,
profiling (15.344 s) : 15344000, 15344000
.   : milestone, 15344000,
tracing (15.041 s) : 15041000, 15041000
.   : milestone, 15041000,
section candidate
no_agent (14.905 s) : 14905000, 14905000
.   : milestone, 14905000,
appsec (14.848 s) : 14848000, 14848000
.   : milestone, 14848000,
iast (18.6 s) : 18600000, 18600000
.   : milestone, 18600000,
iast_GLOBAL (17.802 s) : 17802000, 17802000
.   : milestone, 17802000,
profiling (15.285 s) : 15285000, 15285000
.   : milestone, 15285000,
tracing (14.89 s) : 14890000, 14890000
.   : milestone, 14890000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.723 s [15.723 s, 15.723 s] -
appsec 14.974 s [14.974 s, 14.974 s] -749.0 ms (-4.8%)
iast 18.343 s [18.343 s, 18.343 s] 2.62 s (16.7%)
iast_GLOBAL 18.234 s [18.234 s, 18.234 s] 2.511 s (16.0%)
profiling 15.344 s [15.344 s, 15.344 s] -379.0 ms (-2.4%)
tracing 15.041 s [15.041 s, 15.041 s] -682.0 ms (-4.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.905 s [14.905 s, 14.905 s] -
appsec 14.848 s [14.848 s, 14.848 s] -57.0 ms (-0.4%)
iast 18.6 s [18.6 s, 18.6 s] 3.695 s (24.8%)
iast_GLOBAL 17.802 s [17.802 s, 17.802 s] 2.897 s (19.4%)
profiling 15.285 s [15.285 s, 15.285 s] 380.0 ms (2.5%)
tracing 14.89 s [14.89 s, 14.89 s] -15.0 ms (-0.1%)

@pr-commenter
Copy link

pr-commenter bot commented Aug 22, 2025

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch nikita-tkachenko/junit-6 sarahchen6/experiment
git_commit_date 1754670324 1755891364
git_commit_sha 7c7286a 9871ee0
See matching parameters
Baseline Candidate
ci_job_date 1755892435 1755892435
ci_job_id 1095096698 1095096698
ci_pipeline_id 74529941 74529941
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter
Copy link

pr-commenter bot commented Aug 22, 2025

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch nikita-tkachenko/junit-6 sarahchen6/experiment
git_commit_date 1754670324 1755891364
git_commit_sha 7c7286a 9871ee0
See matching parameters
Baseline Candidate
ci_job_date 1755892475 1755892475
ci_job_id 1095096699 1095096699
ci_pipeline_id 74529941 74529941
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.