Skip to content

Test apm-sdks-benchmarks branch changes#11369

Draft
sarahchen6 wants to merge 1 commit into
sarahchen6/run-startup-alwaysfrom
sarahchen6/test-benchmark-comparisons
Draft

Test apm-sdks-benchmarks branch changes#11369
sarahchen6 wants to merge 1 commit into
sarahchen6/run-startup-alwaysfrom
sarahchen6/test-benchmark-comparisons

Conversation

@sarahchen6
Copy link
Copy Markdown
Contributor

@sarahchen6 sarahchen6 commented May 13, 2026

What Does This Do

This PR tests https://github.com/DataDog/apm-sdks-benchmarks/pull/151 which runs the latest merge-base master agent against the same benchmarks that the PR runs, calculates the % change between the results, and reports it in the PR comment.

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@sarahchen6 sarahchen6 added the tag: do not merge Do not merge changes label May 13, 2026
@sarahchen6 sarahchen6 force-pushed the sarahchen6/test-benchmark-comparisons branch 4 times, most recently from 522be3f to 234690e Compare May 14, 2026 18:20
@DataDog DataDog deleted a comment from dd-octo-sts Bot May 14, 2026
@sarahchen6 sarahchen6 force-pushed the sarahchen6/test-benchmark-comparisons branch from a2458e1 to 234690e Compare May 14, 2026 20:28
@sarahchen6 sarahchen6 force-pushed the sarahchen6/run-startup-always branch from 69bcc95 to 95573aa Compare May 15, 2026 14:57
@sarahchen6 sarahchen6 force-pushed the sarahchen6/test-benchmark-comparisons branch 2 times, most recently from 5ccf89a to 9e3a894 Compare May 15, 2026 18:27
@sarahchen6 sarahchen6 force-pushed the sarahchen6/test-benchmark-comparisons branch from 9e3a894 to 4941d08 Compare May 15, 2026 20:01
@DataDog DataDog deleted a comment from dd-octo-sts Bot May 15, 2026
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 15, 2026

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass
Load 🟢 pass
DaCapo 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Startup Time

Scenario This PR master Change
insecure-bank / iast 14,023 ms 13,938 ms +0.6%
insecure-bank / tracing 12,987 ms 13,001 ms -0.1%
petclinic / appsec 16,431 ms 16,316 ms +0.7%
petclinic / iast 16,453 ms 16,512 ms -0.4%
petclinic / profiling 16,397 ms 16,468 ms -0.4%
petclinic / tracing 15,797 ms 15,102 ms +4.6%

DaCapo Execution Time

Scenario This PR master Change
biojava / appsec 13,646 ms 13,793 ms -1.1%
biojava / tracing 14,490 ms 14,449 ms +0.3%
tomcat / appsec 2.17 ms 2.16 ms +0.6%
tomcat / tracing 1.65 ms 1.64 ms +0.2%

Load Latency (p50)

Scenario This PR master Change
insecure-bank / tracing 0.82 ms 0.79 ms +3.1%
spring-petclinic / tracing 26.04 ms 26.19 ms -0.6%

Commit: 4941d089 · CI Pipeline · Benchmarking Platform UI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tag: do not merge Do not merge changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant