feat(l2): based full-sync #3243
Draft
+4,107
−2,301
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
**Motivation** <!-- Why does this pull request exist? What are its goals? --> This PRs makes sure we batch the insertions of new code into the DB. For that, it adds a new field to the `UpdatesBatch` struct and modifies the relevant functions. <!-- A clear and concise general description of the changes this PR introduces --> <!-- Link to issues: Resolves #111, Resolves #222 --> Closes #3193
**Motivation** Publish the mdbook of this repo (book.toml) to https://docs.ethrex.xyz/ **Description** These changes are to leave the setup like this: * https://docs.ethrex.xyz/ will have the mdbook * https://docs.ethrex.xyz/benchmarks will have the benchmarks graphs * https://docs.ethrex.xyz/flamegraphs will have the flamegraphs
**Motivation** Our L2 documentation lacks a clear structure. **Description** This PR reorganizes our L2 docs, also moving documentation on L2 load-tests under `Developers`->`L2 load-tests`. The rest of the documentation was restructured to a structure like that of other L1 and L2 projects: <img width="297" alt="new structure of L2 docs" src="https://github.com/user-attachments/assets/b9c89a10-c175-4610-b141-3fa4b0097cfb" /> Documentation on smart contracts still needs to be filled and is only acting as a placeholder for now. <!-- Link to issues: Resolves #111, Resolves #222 --> Closes #3174
**Motivation** Our build payload process was not computing and setting the `logs_bloom` field on the block's header, which resulted in other clients rejecting blocks built by us. This came up when testing setting up a localnet with ethrex along with other clients. **Description** <!-- A clear and concise general description of the changes this PR introduces --> <!-- Link to issues: Resolves #111, Resolves #222 --> Closes #issue_number
**Motivation** Fix for #3217 **Description** Fixes lack of permissions for mdbook workflow, and new path to publish L1 block proving benchmark
**Motivation** - Benchmarks are a key piece for measuring performance, the code wasn't very concise so this simplifies it to make further changes that will help us work on performance in LEVM. **Description** Behavior is pretty much the same, the code is just more clear now. Closes #issue_number
…ins empty (#3228) **Motivation** The l2 committer was stuck because it failed when trying to encode state diffs of an account that was initially empty, remained empty after the transaction so the AccountUpdate was completely empty and state diff creation failed with `StateDiffError::EmptyAccountDiff` **Description** - `LEVM::get_state_transitions` now checks if the account was initially empty, in case it was and it remains empty after the transaction do not count it as an AccountUpdate
**Motivation** `EstimateGasPriceError` is actually an error triggered in `estimate_gas`. **Description** Renames `EstimateGasPriceError` to `EstimateGasError` Closes None
…3192) **Motivation** In the `ProofCoordinator`'s `handle_listens` function, a semaphore is used to limit the number of concurrent connections. This constraint can be removed since it is not necessary and adds complexity to the codebase. **Description** Removes the semaphore and the concurrent connections limit from the `ProofCoordinator`'s `handle_listens` function. It also adds the `ConnectionHandler::spawn` method to encapsulate the `ConnectionHandler` startup logic.
…rt for ContextResult in some places (#3134) **Motivation** - `ExecutionResult` isn't accurate for interaction between callframes so the goal is to replace it for `ContextResult` that has the necessary data. Also, `Substate` should be as specified in Yellow Paper. **Description** - Add logs to substate and remove them from the callframe. They belong to the substate according to section 6.1 of the [yellow paper](https://ethereum.github.io/yellowpaper/paper.pdf). - Replace usage of ExecutionReport in callframes execution for ContextResult. The former contained data that wasn't necessary and caused a little bit of confusion. In ContextResult we have only the data we need: `gas_used`, `output` and `result`. - Move `is_create` logic to `CallFrame`. So now it is not `create_op_called`, it is `is_create` and it takes into account external transactions, not only internal `create`. - Make functions `handle_opcode_result()` and `handle_opcode_error()` prettier. - `finalize_execution()` now returns an `ExecutionReport` given a `ContextResult` - Refactor `increase_consumed_gas()`, behavior is still the same but logic before was kinda repetitive. <!-- Link to issues: Resolves #111, Resolves #222 --> Closes #3045 --------- Co-authored-by: Javier Rodríguez Chatruc <[email protected]> Co-authored-by: juanbono <[email protected]> Co-authored-by: fedacking <[email protected]>
**Motivation** The default value of `proof-coordinator.dev-mode` is set to true. This means the only way to set it to false is through the environment variable `ETHREX_PROOF_COORDINATOR_DEV_MODE`. This is also inconsistent with the rest of the parameters, where we set dev values only in the Makefile. **Description** Changes the default value of `dev-mode` to false. Closes None
**Motivation** As outlined in #3124, sometimes a committed batch can't be verified or the operator wants to prevent it from going though. **Description** This PR implements a `revertBatch` function that allows reverting back to any batch, as long as no verified batches are being discarded. There's also a l2 CLI subcommand, revert-batch that lets you revert a batch and remove it from the local database. Usage on local network: ``` PRIVATE_KEY=key cargo run --features l2,rollup_storage_libmdbx -- l2 revert-batch \ <batch to revert to> <OnChainProposer address> \ --datadir dev_ethrex_l2 --network test_data/genesis-l2.json ``` Closes #3124
Lines of code reportTotal lines added: Detailed view |
**Motivation** Most Ethereum clients let you speed up or overwrite transactions by accepting new transactions with the same nonce but higher fees. This PR adds validations similar to what [Geth does](https://github.com/ethereum/go-ethereum/blob/09289fd154a45420ec916eb842bfb172df7e0d83/core/txpool/legacypool/list.go#L298-L345) but without the `PriceBump` minimum bump percentage **Description** - for eip-1559 check that both `max_fee_per_gas` and `max_priority_fee_per_gas` are greater in the new tx - for legacy tx check that new `gas_price` is greater in the new tx - for eip-4844 txs check that `max_fee_per_gas`, `max_priority_fee_per_gas` and `max_fee_per_blob_gas` are grater in the new tx **How to test** - Send a tx with very low gas price ```shell rex send --gas-price 1 --priority-gas-price 1 --rpc-url http://localhost:1729 0x2B29Bea668B044b2b355C370f85b729bcb43EC40 100000000000000 0x8f87d3aca3eff8132256f69e17df5ba3c605e1b5f4e2071d56f7e6cd66047cc2 ``` - Check tx pool the you should see something like `"maxPriorityFeePerGas":"0x1","maxFeePerGas":"0x1","gasPrice":"0x1"` the tx will probably get stuck ``` curl 'http://localhost:1729' --data '{ "id": 1, "jsonrpc": "2.0", "method": "txpool_content", "params": [] }' -H 'accept: application/json' -H 'Content-Type: application/json' ``` - Send tx with higher gas ```shell rex send --gas-price 100000000 --priority-gas-price 100000000 --rpc-url http://localhost:1729 0x2B29Bea668B044b2b355C370f85b729bcb43EC40 100000000000000 0x8f87d3aca3eff8132256f69e17df5ba3c605e1b5f4e2071d56f7e6cd66047cc2 ``` - Check that the tx pool you should see something like `"maxPriorityFeePerGas":"0x5f5e100","maxFeePerGas":"0x5f5e100","gasPrice":"0x5f5e100"` ```shell curl 'http://localhost:1729' --data '{ "id": 1, "jsonrpc": "2.0", "method": "txpool_content", "params": [] }' -H 'accept: application/json' -H 'Content-Type: application/json' ```
Add roadmap
**Motivation** The LEVM CI workflow in pr-main_levm.yaml that runs EF state tests should fail with an exit code if a test fails. **Description** This pr introduces a new `EFTestRunnerError::TestsFailed` error to use when there's a report of a test failing. This error is thrown under the `summary` flag, which is the one used in the target the CI job executes: `make run-evm-ef-tests-ci`. So whenever there is any failing tests, the introduced code should print the EFTestReport and then finish with the `EFTestRunnerError::TestsFailed` error. Note: The `summary` flag is used in other targets as well, so the previously described behavior is being implemented for other targets too. The ef-test-main job in pr-main_levm has also been refactored, I dropped steps "Check EF-TESTS from Paris to Prague is 100%" and "Check EF-TESTS status is 100%" since now in the case any test fails, the CI job exits with an error code and outputs the failing tests in the console. In this pr there are some commits with a hardcoded error with the intentions of having the LEVM CI workflow fail on purpose and check the console output is the one expected. [Here](https://github.com/lambdaclass/ethrex/actions/runs/15738130731/job/44356244936) is a failing workflow execution under this circumstances to see. (The underscore line above "Failed tests" was removed on a later commit.) Closes #2887
**Motivation** We want to include diagrams in the mdbook. The easiest way to manage diagrams with `git` is to declare them with `mermaid`. **Description** This PR adds [the `mdbook-mermaid` preprocessor](https://github.com/badboy/mdbook-mermaid), which automatically renders the mermaid diagrams in our docs. As part of this, it also adds make targets to automatically install preprocessors/backends, and to generate the files required by `mdbook-mermaid`. <img width="836" alt="example mermaid diagram in the L2 docs" src="https://github.com/user-attachments/assets/d14d57f4-4c73-4c99-82e3-281f1693ee84" />
**Motivation** When running a localnet with kurtosis the ethrex client wasn't exposing a metrics port. **Description** To expose the metrics port, the ETHEREUM_PACKAGE_REVISION in the Makefile was updated to the latest commit in our fork of ethereum-package. Additionally, the metrics feature flag was enabled when building the Docker image (without it, metrics won't work). The ethereum_metrics_exporter_enabled setting was also enabled for all participants in the ethrex-only localnet. With these changes, we are now able to use metrics with ethrex clients. <!-- Link to issues: Resolves #111, Resolves #222 --> Closes #3213
…s file (#3111) **Motivation** <!-- Why does this pull request exist? What are its goals? --> When using a pre-merge genesis.json for importing blocks, which is not supported by ethrex, the error received was `ParentNotFound`, which doesn't explain the real problem. **Description** <!-- A clear and concise general description of the changes this PR introduces --> Before merging blocks the genesis.json fork is checked, in case that its pre Paris return a custom error message. For doing this new checks were added to the `fork()` function. <!-- Link to issues: Resolves #111, Resolves #222 --> Closes #3102 --------- Co-authored-by: Martin Paulucci <[email protected]>
**Motivation** This was failing on multiple PRs because the ethrex_dev image was not being built. The difference between the failing job and the others (which were succeeding) is the runner (larger_runners). Maybe that has something to do with it. **Description** - adds a step to build ethrex_dev explicitly
… names (#3261) **Motivation** We can avoid a double lookup with `storage_original_values` by using a tuple. I found gen_db field names to be a bit confusing and not giving useful information, "immutable_cache" is not really a cache but a store for initial account states, so i changed it. Same with cache. **Description** <!-- A clear and concise general description of the changes this PR introduces --> <!-- Link to issues: Resolves #111, Resolves #222 --> Closes #issue_number
**Motivation** According to stats from @azteca1998 PUSH2 and PUSH1 are widely used: ``` Loaded 903636264 rows (3447.10MiB) Stats (of 903636264 records): 0xf1: count= 730979 t_min= 2278 t_max=1512728 t_avg=110877.43 t_acc=81049072024 CALL 0x61: count=131856777 t_min= 136 t_max= 549032 t_avg= 189.29 t_acc=24959614846 PUSH2 0x56: count= 78745029 t_min= 170 t_max=1488792 t_avg= 243.75 t_acc=19194034756 JUMP 0x60: count= 86327863 t_min= 136 t_max= 837080 t_avg= 199.78 t_acc=17246262544 PUSH1 0x5b: count=107216057 t_min= 102 t_max= 267308 t_avg= 159.43 t_acc=17093508806 JUMPDEST 0x50: count= 86546732 t_min= 102 t_max= 353260 t_avg= 174.49 t_acc=15101132640 POP 0x57: count= 53096953 t_min= 102 t_max=1382576 t_avg= 233.40 t_acc=12393069292 JUMPI 0x81: count= 55585321 t_min= 102 t_max= 267410 t_avg= 192.79 t_acc=10716509980 DUP2 0x01: count= 56493418 t_min= 102 t_max=1431060 t_avg= 189.52 t_acc=10706399944 ADD 0x91: count= 31380921 t_min= 102 t_max= 146030 t_avg= 205.38 t_acc= 6444862520 SWAP2 ``` Furthermore i keep seeing `U256::from_big_endian` taking quite some time on samply so I made specialized PUSH1 and PUSH2 implementations that avoid that, also using fixed size arrays. Benchmarks: Hoodi 11k: main 9m10.471s pr 8m25.933s **Description** <!-- A clear and concise general description of the changes this PR introduces --> <!-- Link to issues: Resolves #111, Resolves #222 --> Closes #issue_number # Benchmark Results Comparison #### Benchmark Results: Factorial | Command | Mean [ms] | Min [ms] | Max [ms] | Relative | |:---|---:|---:|---:|---:| | `levm_Factorial_pr` | 634.2 ± 7.3 | 629.6 | 654.2 | 2.71 ± 0.04 | | `levm_Factorial` | 726.1 ± 5.2 | 722.5 | 740.1 | 3.11 ± 0.03 | | `levm_FactorialRecursive_pr` | 3.567 ± 0.021 | 3.541 | 3.604 | 2.22 ± 0.05 | | `levm_FactorialRecursive` | 3.828 ± 0.035 | 3.775 | 3.889 | 2.39 ± 0.03 | | `levm_Fibonacci_pr` | 629.2 ± 6.4 | 625.7 | 646.9 | 2.99 ± 0.03 | | `levm_Fibonacci` | 727.7 ± 6.5 | 722.3 | 743.9 | 3.47 ± 0.03 | | `levm_ManyHashes_pr` | 14.9 ± 0.2 | 14.7 | 15.3 | 1.70 ± 0.03 | | `levm_ManyHashes` | 16.3 ± 0.1 | 16.2 | 16.4 | 1.87 ± 0.02 | | `levm_BubbleSort_pr` | 5.065 ± 0.023 | 5.034 | 5.107 | 1.58 ± 0.01 | | `levm_BubbleSort` | 5.508 ± 0.035 | 5.489 | 5.603 | 1.71 ± 0.02 | | `levm_ERC20Transfer_pr` | 461.5 ± 1.3 | 459.7 | 463.4 | 1.87 ± 0.03 | | `levm_ERC20Transfer` | 487.9 ± 2.4 | 484.1 | 491.0 | 1.99 ± 0.01 | | `levm_ERC20Mint_pr` | 306.8 ± 8.9 | 300.1 | 328.5 | 2.22 ± 0.07 | | `levm_ERC20Mint` | 320.1 ± 1.5 | 317.9 | 322.6 | 2.31 ± 0.05 | | `levm_ERC20Approval_pr` | 1.779 ± 0.023 | 1.763 | 1.838 | 1.69 ± 0.02 | | `levm_ERC20Approval` | 1.850 ± 0.011 | 1.837 | 1.873 | 1.76 ± 0.02 |  According to the samply this makes op_push nearly negligible (from 30% to 0%) --------- Co-authored-by: Jeremías Salomón <[email protected]>
**Motivation** On x86_64, rust has a harder time when the match is used, 2 things happen: With match; - Apparently it also uses a lookup table internally but it doesn't have as much "info" about what we doing than when doing it manually, for example the function has an extra xor instruction, it also looks like it has more trouble inlining the From Without match: - No unneeded xor instruction - Easier to inline for the compiler (as seen on the godbolt url), this avoids a full function call. Godbolt: https://godbolt.org/z/eG8M1jz3M Closes #2896 Should close #2896
Add type 4 transaction validations to validations.md docs Closes #2545
**Motivation** Improve write up of the L1 roadmap
**Motivation** Go back to the normal GitHub runners, instead of larger runners, because the disk size constraint has been removed from this CI job **Description** * Changes `runs-on:` from `integration-test` job in `pr-main_l2.yaml` workflow. * Removes actionlint label * Removes step and comment related to using the larger runners --------- Co-authored-by: Javier Chatruc <[email protected]> Co-authored-by: Javier Rodríguez Chatruc <[email protected]>
**Description** This PR changes the mdbook build action to use the `docs-deps` and `docs` targets from the makefile to install dependencies and build documentation, instead of manually doing so like until now.
**Motivation** We want to be able to decode ABI-packed data. **Description** This PR copies the implementation [made for rex](lambdaclass/rex#134), with two slicing operations fixed to avoid panicking. --------- Co-authored-by: Avila Gastón <[email protected]>
#3180 happened again Co-authored-by: LeanSerra <[email protected]> Co-authored-by: Javier Rodríguez Chatruc <[email protected]>
**Motivation** Roadmap was hard to follow through in a fast read. **Description** Rearranged the roadmap items to group "In Progress" and "Planned" tasks properly.
**Motivation** In preparation to support more complex bridging (such as of ERC20 assets), we want to use generic L2->L1 messaging primitives that can be easily extended and reused. **Description** This replaces withdrawals with a new type L1Message, and has the bridge make use of them. - Allows for multiple messages per transaction - Allows for arbitrary data to be sent. Hashed for easier handling. --------- Co-authored-by: Tomás Grüner <[email protected]> Co-authored-by: Avila Gastón <[email protected]>
**Motivation** Our `transaction_tracker` metric is reset every time the node is restarted. **Description** - Uses the `increase()` function in Grafana to avoid resetting the counter on node restarts. - Initializes each possible value to 0 when starting the metrics to properly calculate increments. - Splits the Transaction panel into two: `Transactions` and `Transaction Errors`. - Inverts the colors in `Gas Limit Usage` and `Blob Gas Usage`. - Pushes transaction metrics only after closing the block in L2, since transactions may be added or removed depending on the state diff size. **Last Blob Usage**: Before | After <img width="312" alt="image" src="https://github.com/user-attachments/assets/cd8e5471-3fa9-491b-93c0-10cf24da663c" /> <img width="324" alt="image" src="https://github.com/user-attachments/assets/1fe9b992-1d05-4269-86dd-78ec1f885be0" /> **Transactions** <img width="700" alt="image" src="https://github.com/user-attachments/assets/ff785f62-fc07-406f-8e8e-4d0f2b4d9aa1" /> **Transaction Errors** <img width="694" alt="image" src="https://github.com/user-attachments/assets/146d46b0-c22b-4ff4-969d-a57acdc7916b" /> ### How to test 1. Start an L2 node with metrics enabled: ```bash cd ethrex/crates/l2 make init ``` 2. Go to `http://localhost:3802/` to watch the Grafana dashboard. 3. Restart the node and check that the `Transactions` panel is not reset. ```bash crtl + c make init-l2-no-metrics ``` 4. Modify `apply_plain_transactions` in `ethrex/crates/blockchain/payload.rs:543` to generate some errors: ```Rust pub fn apply_plain_transaction( head: &HeadTransaction, context: &mut PayloadBuildContext, ) -> Result<Receipt, ChainError> { use std::time::{SystemTime, UNIX_EPOCH}; let now = SystemTime::now().duration_since(UNIX_EPOCH).unwrap(); let seed = (now.as_secs() ^ now.subsec_nanos() as u64) as usize; match seed % 5 { 1 => Err(ChainError::ParentNotFound), 2 => Err(ChainError::ParentStateNotFound), 3 => Err(ChainError::InvalidTransaction("tx error".into())), 4 => Err(ChainError::WitnessGeneration("witness failure".into())), _ => Err(ChainError::Custom("custom error".into())), } } ``` 5. Restart the node and send some transactions: ```bash cd ethrex cargo run --release --manifest-path ./tooling/load_test/Cargo.toml -- -k ./test_data/private_keys.txt -t eth-transfers -n http://localhost:1729 ``` if necessary run `ulimit -n 65536` before the command. Closes None
**Motivation** Make the "build block" benchmark run in the CI. --------- Co-authored-by: Martin Paulucci <[email protected]>
**Motivation** We are using a specific `aligned-sdk` commit. Now that we've bumped the SP1 version to `v5.0.0`, we can use the latest release. **Description** - Uses the latest release of the `aligned-sdk`. - Refactors `estimate_gas` since some clients don't allow empty `blobVersionedHashes`, and our deployer doesn't work with `ethereum-package`. - Adds a guide on how to run an Aligned dev environment. ## How to test Read the new section `How to Run Using an Aligned Dev Environment` in `docs/l2/aligned_mode.md`. Closes #3169
Benchmark Results ComparisonPR ResultsBenchmark Results: Factorial
Benchmark Results: Factorial - Recursive
Benchmark Results: Fibonacci
Benchmark Results: ManyHashes
Benchmark Results: BubbleSort
Benchmark Results: ERC20 - Transfer
Benchmark Results: ERC20 - Mint
Benchmark Results: ERC20 - Approval
Main ResultsBenchmark Results: Factorial
Benchmark Results: Factorial - Recursive
Benchmark Results: Fibonacci
Benchmark Results: ManyHashes
Benchmark Results: BubbleSort
Benchmark Results: ERC20 - Transfer
Benchmark Results: ERC20 - Mint
Benchmark Results: ERC20 - Approval
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Description
Closes #issue_number