Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the benchmark CI #222

Merged
merged 6 commits into from
Jan 10, 2025
Merged

Improve the benchmark CI #222

merged 6 commits into from
Jan 10, 2025

Conversation

MahdiBM
Copy link
Contributor

@MahdiBM MahdiBM commented Jan 9, 2025

No description provided.

Copy link

github-actions bot commented Jan 9, 2025

Benchmark Report

✅ Pull request has no significant performance differences ✅

Click to expand comparison result

Benchmark check running at 2025-01-10 13:56:04 UTC

The baseline mmbm-benchmark-ci is EQUAL to the defined thresholds.

Click to expand benchmark result

Baseline mmbm-benchmark-ci

Host 477dbef13a25 with 2 aarch64 processors with 3 GB memory, running:
#23-Ubuntu SMP Mon Dec  9 23:51:16 UTC 2024

Signing

ES256

Metric p0 p25 p50 p75 p90 p99 p100 Samples
Malloc (total) * 169 169 169 203 271 373 509 2162
Memory (resident peak) (M) 26 27 27 27 27 27 27 2162

EdDSA

Metric p0 p25 p50 p75 p90 p99 p100 Samples
Malloc (total) * 117 117 117 117 117 117 117 3676
Memory (resident peak) (M) 26 26 26 26 27 27 27 3676

RSA

Metric p0 p25 p50 p75 p90 p99 p100 Samples
Malloc (total) * 239 239 239 248 248 248 248 612
Memory (resident peak) (M) 26 27 27 27 27 27 27 612

TokenLifecycle

ES256 Generated

Metric p0 p25 p50 p75 p90 p99 p100 Samples
Malloc (total) * 355 355 355 389 457 559 729 1655
Memory (resident peak) (M) 27 27 27 27 27 27 27 1655

ES256 PEM

Metric p0 p25 p50 p75 p90 p99 p100 Samples
Malloc (total) * 373 373 373 373 373 373 373 2414
Memory (resident peak) (M) 27 27 27 27 27 27 27 2414

EdDSA Coordinates

Metric p0 p25 p50 p75 p90 p99 p100 Samples
Malloc (total) * 323 323 323 323 323 323 323 2213
Memory (resident peak) (M) 27 27 27 27 27 27 27 2213

EdDSA Generated

Metric p0 p25 p50 p75 p90 p99 p100 Samples
Malloc (total) * 291 291 291 291 291 291 291 2333
Memory (resident peak) (M) 26 27 27 27 27 27 27 2333

RSA PEM

Metric p0 p25 p50 p75 p90 p99 p100 Samples
Malloc (total) * 436 436 436 445 445 445 445 573
Memory (resident peak) (M) 27 27 27 27 27 27 27 573

Verifying

ES256

Metric p0 p25 p50 p75 p90 p99 p100 Samples
Malloc (total) * 263 263 263 263 263 263 263 3142
Memory (resident peak) (M) 26 27 27 27 27 27 27 3142

EdDSA

Metric p0 p25 p50 p75 p90 p99 p100 Samples
Malloc (total) * 251 251 251 251 251 251 251 2737
Memory (resident peak) (M) 27 27 27 27 27 27 27 2737

RS256

Metric p0 p25 p50 p75 p90 p99 p100 Samples
Malloc (total) * 257 257 257 257 257 257 257 3274
Memory (resident peak) (M) 27 27 27 27 27 27 27 3274

@MahdiBM MahdiBM force-pushed the mmbm-benchmark-ci branch from af7f7e0 to 99bcb7c Compare January 9, 2025 13:28
Copy link
Member

@gwynne gwynne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A zillion little nits mostly about how to use Bash.

.github/workflows/benchmark.yml Outdated Show resolved Hide resolved
.github/workflows/benchmark.yml Show resolved Hide resolved
.github/workflows/benchmark.yml Outdated Show resolved Hide resolved
.github/workflows/benchmark.yml Outdated Show resolved Hide resolved
.github/workflows/benchmark.yml Show resolved Hide resolved
.github/workflows/benchmark.yml Outdated Show resolved Hide resolved
.github/workflows/benchmark.yml Outdated Show resolved Hide resolved
.github/workflows/benchmark.yml Outdated Show resolved Hide resolved
.github/workflows/benchmark.yml Outdated Show resolved Hide resolved
.github/workflows/benchmark.yml Outdated Show resolved Hide resolved
@MahdiBM MahdiBM force-pushed the mmbm-benchmark-ci branch from ee82025 to c6209b3 Compare January 9, 2025 14:34
@MahdiBM MahdiBM requested a review from gwynne January 9, 2025 15:12
@MahdiBM MahdiBM force-pushed the mmbm-benchmark-ci branch 2 times, most recently from 3787c39 to 9ce35f3 Compare January 9, 2025 16:36
.github/workflows/benchmark.yml Outdated Show resolved Hide resolved
.github/workflows/benchmark.yml Outdated Show resolved Hide resolved
.github/workflows/benchmark.yml Show resolved Hide resolved
@MahdiBM MahdiBM force-pushed the mmbm-benchmark-ci branch from 7be528b to 546a8b1 Compare January 9, 2025 17:24
Copy link

codecov bot commented Jan 10, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.79%. Comparing base (e447bee) to head (4edf96c).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #222   +/-   ##
=======================================
  Coverage   83.79%   83.79%           
=======================================
  Files          56       56           
  Lines        1493     1493           
=======================================
  Hits         1251     1251           
  Misses        242      242           

@ptoffy ptoffy self-requested a review January 10, 2025 10:39
Comment on lines +15 to +16
# Run the job only if it's a manual workflow dispatch, or if this event is a pull-request approval event, or if someone has rerun the job.
if: github.event_name == 'workflow_dispatch' || github.event.review.state == 'approved' || github.run_attempt > 1
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated this condition @gwynne @ptoffy to always run the job if someone has rerun it.

Comment on lines 9 to +12
paths:
- 'Sources/*.swift'
- .github/workflows/benchmark.yml
- Sources/*.swift
- Benchmarks/*.swift
- .github/workflows/benchmark.yml
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added - Benchmarks/*.swift too

@ptoffy ptoffy merged commit bbdc56d into main Jan 10, 2025
14 checks passed
@ptoffy ptoffy deleted the mmbm-benchmark-ci branch January 10, 2025 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants