-
Notifications
You must be signed in to change notification settings - Fork 171
refactor(benchmark): update expected gas usage rule for flexibility #2155
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Works from my side. From the call we had a couple weeks ago on the 1st of September with Guilliaume my expectation was that we'd make this change. LGTM!
cc-ing @marioevz for when he's back - happy to merge nonetheless!
Tagged @fselmo for another opinion - if all good I say we should merge. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me. We will have to be very diligent about checking the logic of the tests that set this. I added one nit to think about but this is not a blocker.
74504c1
to
3b7db94
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm!
🗒️ Description
Currently, we enforce verification of
expected_benchmark_gas_used
, which ensures that no invalid operation terminates execution prematurely. This PR relaxes that rule to give benchmark developers more flexibility.For example, when benchmarks use while-loops or conditional comparisons, calculating the exact gas usage becomes difficult. In such cases, developers may end up spending more effort on tracking precise gas costs than on the test logic itself. Moreover, not all benchmark cases are intended to fully exhaust the block’s gas like it does in gas-limit-testing.
That said, this change does not mean future benchmarks can skip gas accounting altogether. This exception should only apply when gas usage is inherently hard to predict. By default, reviewers must still verify that
expected_benchmark_gas_used
is correctly set.This is initially discussed in PR #2090, although PR created, it is still open to discussion!
🔗 Related Issues or PRs
PR #2090
Comment: #2090 (comment)
✅ Checklist
tox
checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:uvx --with=tox-uv tox -e lint,typecheck,spellcheck,markdownlint
type(scope):
.mkdocs serve
locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.@ported_from
marker.