-
Notifications
You must be signed in to change notification settings - Fork 1
Description
Versions
- Polkadot SDK commit
1a512570552119a49a8ecb2abfb7021954c4422d
- Differential Testing tool commit
8b1afc36a32bb27e4d9680f08afe97e1946b0269
- Resolc Compiler Tests commit
4372389f90af54b5c2543050f228a6ca2f44e787
- Targets: Geth's EVM and Revive Dev Node's PolkaVM
Summary
When running the differential testing tool with Geth VS Revive Dev Node we have noticed that there are certain cases where execution fails with "Invalid Transaction" when submitting the transaction to Kitchensink.
Alex previously mentioned that this is related to the gas mapping.
Description
When looking at the logs of the dev-node and eth-rpc's started by the tool we see the following in there:
2025-08-26 18:43:18.631 DEBUG tokio-runtime-worker eth-rpc: submit call failed: SubxtError(Rpc(ClientError(User(UserError { code: 1010, message: "Invalid Transaction", data: Some(RawValue("Inability to pay some fees (e.g. account balance too low)")) }))))
Therefore, it appears that the "invalid transaction" error is actually related to gas, the gas model, or to fees.
Steps to Reproduce
The following command can be used to reproduce this through the differential testing tool. Note that the following command will clone and rebuild the polkadot-sdk at the commit specified in the versions above.
(
set -euo pipefail
POLKADOT_REPO="https://github.com/paritytech/polkadot-sdk/"
POLKADOT_COMMIT="1a512570552119a49a8ecb2abfb7021954c4422d"
mkdir -p test
cd test
if [[ ! -d revive-differential-tests/.git ]]; then
git clone https://github.com/paritytech/revive-differential-tests
fi
git -C revive-differential-tests fetch --all --prune
git -C revive-differential-tests checkout 8b1afc36a32bb27e4d9680f08afe97e1946b0269
git -C revive-differential-tests reset --hard 8b1afc36a32bb27e4d9680f08afe97e1946b0269
if [[ ! -d resolc-compiler-tests/.git ]]; then
git clone https://github.com/paritytech/resolc-compiler-tests
fi
git -C resolc-compiler-tests fetch --all --prune
git -C resolc-compiler-tests checkout 4372389f90af54b5c2543050f228a6ca2f44e787
git -C resolc-compiler-tests reset --hard 4372389f90af54b5c2543050f228a6ca2f44e787
echo
printf "Path to existing polkadot-sdk (leave empty to clone %s): " "${POLKADOT_COMMIT}"
IFS= read -r POLKADOT_PATH_INPUT || true
POLKADOT_PATH_INPUT="${POLKADOT_PATH_INPUT/#\~/$HOME}"
if [[ -z "${POLKADOT_PATH_INPUT}" ]]; then
echo "Cloning polkadot-sdk @ ${POLKADOT_COMMIT}..."
if [[ ! -d polkadot-sdk/.git ]]; then
git clone "${POLKADOT_REPO}" polkadot-sdk
fi
git -C polkadot-sdk fetch --all --prune
git -C polkadot-sdk checkout "${POLKADOT_COMMIT}"
git -C polkadot-sdk reset --hard "${POLKADOT_COMMIT}"
POLKADOT_PATH="./polkadot-sdk"
else
POLKADOT_PATH="${POLKADOT_PATH_INPUT}"
if [[ ! -d "${POLKADOT_PATH}" ]] || [[ ! -f "${POLKADOT_PATH}/Cargo.toml" ]]; then
echo "Error: '${POLKADOT_PATH}' doesn't look like a polkadot-sdk checkout (missing directory or Cargo.toml)." >&2
exit 1
fi
echo "Using user-provided polkadot-sdk at: ${POLKADOT_PATH}"
fi
cargo build --release --manifest-path ./revive-differential-tests/Cargo.toml
cargo build --release --manifest-path "${POLKADOT_PATH}/Cargo.toml" \
--package pallet-revive-eth-rpc \
--package revive-dev-node
find "./revive-differential-tests/target/release/" -type d -mindepth 1 -prune -o -type f \( -perm -100 -o -perm -010 -o -perm -001 \) -exec cp -p {} "."/ \;
find "${POLKADOT_PATH}/target/release/" -type d -mindepth 1 -prune -o -type f \( -perm -100 -o -perm -010 -o -perm -001 \) -exec cp -p {} "."/ \;
cp ./revive-differential-tests/genesis.json .
mkdir -p workdir
cat > corp.json <<'EOF'
{
"name": "MatterLabs Solidity Simple, Complex, and Semantic Tests",
"path": "./resolc-compiler-tests/fixtures/solidity/complex/interface_casting/test.json"
}
EOF
RUST_LOG="info" ./retester \
--corpus "corp.json" \
--workdir "workdir" \
--number-of-nodes 5 \
--kitchensink "./substrate-node" \
--revive-dev-node "./revive-dev-node" \
--eth_proxy "./eth-rpc" \
> logs.log \
2> output.log
)
After this finishes running look at the ./test/logs.log
for the logs and ./test/output.log
for the tool's output and status of each test.
If you want a command to reproduce all of the invalid transaction errors then you can use the following:
Command to run all tests that fail with Invalid Transaction
(
set -euo pipefail
POLKADOT_REPO="https://github.com/paritytech/polkadot-sdk/"
POLKADOT_COMMIT="1a512570552119a49a8ecb2abfb7021954c4422d"
mkdir -p test
cd test
if [[ ! -d revive-differential-tests/.git ]]; then
git clone https://github.com/paritytech/revive-differential-tests
fi
git -C revive-differential-tests fetch --all --prune
git -C revive-differential-tests checkout 8b1afc36a32bb27e4d9680f08afe97e1946b0269
git -C revive-differential-tests reset --hard 8b1afc36a32bb27e4d9680f08afe97e1946b0269
if [[ ! -d resolc-compiler-tests/.git ]]; then
git clone https://github.com/paritytech/resolc-compiler-tests
fi
git -C resolc-compiler-tests fetch --all --prune
git -C resolc-compiler-tests checkout 4372389f90af54b5c2543050f228a6ca2f44e787
git -C resolc-compiler-tests reset --hard 4372389f90af54b5c2543050f228a6ca2f44e787
echo
printf "Path to existing polkadot-sdk (leave empty to clone %s): " "${POLKADOT_COMMIT}"
IFS= read -r POLKADOT_PATH_INPUT || true
POLKADOT_PATH_INPUT="${POLKADOT_PATH_INPUT/#\~/$HOME}"
if [[ -z "${POLKADOT_PATH_INPUT}" ]]; then
echo "Cloning polkadot-sdk @ ${POLKADOT_COMMIT}..."
if [[ ! -d polkadot-sdk/.git ]]; then
git clone "${POLKADOT_REPO}" polkadot-sdk
fi
git -C polkadot-sdk fetch --all --prune
git -C polkadot-sdk checkout "${POLKADOT_COMMIT}"
git -C polkadot-sdk reset --hard "${POLKADOT_COMMIT}"
POLKADOT_PATH="./polkadot-sdk"
else
POLKADOT_PATH="${POLKADOT_PATH_INPUT}"
if [[ ! -d "${POLKADOT_PATH}" ]] || [[ ! -f "${POLKADOT_PATH}/Cargo.toml" ]]; then
echo "Error: '${POLKADOT_PATH}' doesn't look like a polkadot-sdk checkout (missing directory or Cargo.toml)." >&2
exit 1
fi
echo "Using user-provided polkadot-sdk at: ${POLKADOT_PATH}"
fi
cargo build --release --manifest-path ./revive-differential-tests/Cargo.toml
cargo build --release --manifest-path "${POLKADOT_PATH}/Cargo.toml" \
--package pallet-revive-eth-rpc \
--package revive-dev-node
find "./revive-differential-tests/target/release/" -type d -mindepth 1 -prune -o -type f \( -perm -100 -o -perm -010 -o -perm -001 \) -exec cp -p {} "."/ \;
find "${POLKADOT_PATH}/target/release/" -type d -mindepth 1 -prune -o -type f \( -perm -100 -o -perm -010 -o -perm -001 \) -exec cp -p {} "."/ \;
cp ./revive-differential-tests/genesis.json .
mkdir -p workdir
cat > corp.json <<EOF
{
"name": "MatterLabs Solidity Simple, Complex, and Semantic Tests",
"paths": [
"./resolc-compiler-tests/fixtures/solidity/complex/create/create2_chain/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/create/create2_many/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/create/create_chain/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/create/create_many/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/immutable_delegate_call/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/indirect_recursion_fact/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/interface_casting/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/invalid_signature/require/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/invalid_signature/simple/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/nested_calls/staticcall_call/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/nested_calls/staticcall_staticcall/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/solidity_by_example/simple/import/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/solidity_by_example/simple/library/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/solidity_by_example/simple/try_catch/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/try_catch/create/complex/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/try_catch/create/custom_error/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/try_catch/create/error/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/try_catch/create/panic/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/value/delegatecall_call/test.json",
"./resolc-compiler-tests/fixtures/solidity/complex/value/delegatecall_delegatecall/test.json",
"./resolc-compiler-tests/fixtures/solidity/simple/conditional/require.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/constructor/simple.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/create/return_data_clear.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/error/default.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/error/revert1.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/error/revert2.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/error/revert3.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/error/revert4.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/error/revert5.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/function/function_type/f1.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/function/function_type/f3.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/function/function_type/f4.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/function/function_type/f5.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/function/function_type/f6.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/gas_value/gas_value_order1.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/gas_value/gas_value_order2.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/gas_value/value_as_function1.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/gas_value/value_as_function2.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/gas_value/value_as_function3.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/internal_function_pointers/calling_uninitialized_function_in_detail.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/internal_function_pointers/calling_uninitialized_function_through_array.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/internal_function_pointers/function_delete_stack.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/internal_function_pointers/invalidInConstructor.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/internal_function_pointers/invalidStoredInConstructor.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/internal_function_pointers/mixed_features_2.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/internal_function_pointers/mixed_features_3.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/internal_function_pointers/store2.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/internal_function_pointers/storeInConstructor.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/modular/addmod.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/modular/mulmod.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/addition_i128.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/addition_i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/addition_u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/division_i128.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/division_i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/division_u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/exponentiation_i128_const.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/exponentiation_i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/exponentiation_u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/exponentiation_u8_const.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/multiplication_i128.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/multiplication_i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/multiplication_u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/remainder_i128.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/remainder_i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/remainder_u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/subtraction_i128.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/subtraction_i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/arithmetic/subtraction_u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/addition_i128.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/addition_i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/addition_u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/division_i128.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/division_i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/division_u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/multiplication_i128.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/multiplication_i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/multiplication_u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/remainder_i128.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/remainder_i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/remainder_u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/subtraction_i128.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/subtraction_i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/assignment/subtraction_u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/casting/i16_to_i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/operator/casting/simple_casting.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/i128.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/i16.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/i248.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/i256.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/i32.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/i64.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/u16.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/u248.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/u256.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/u32.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/u64.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/negative/u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/overflow_decrement_increment.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/i128.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/i16.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/i248.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/i256.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/i32.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/i64.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/i8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/u16.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/u248.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/u256.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/u32.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/u64.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/positive/u8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/overflow/underflow_increment_decrement.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/pointer/large_offset.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/solidity_by_example/simple/array.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/solidity_by_example/simple/enum.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/solidity_by_example/simple/error.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/solidity_by_example/simple/error_account.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/solidity_by_example/simple/function_modifier.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/stack_overflow.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/try_catch/event_instead_function.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/try_catch/external_call_as_arg.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/try_catch/invalid_return.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/try_catch/not_enough_balance.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/try_catch/not_matching_interface.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/try_catch/not_payable.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/try_catch/signature_collision1.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/try_catch/signature_collision2.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/try_catch/this_in_constructor.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/yul_instructions/invalid.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/yul_instructions/mload.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/yul_instructions/mstore.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/yul_instructions/mstore8.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/yul_instructions/returndatacopy.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/yul_instructions/revert.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/yul_semantic/for.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/yul_semantic/function_definitions.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/yul_semantic/if.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/yul_semantic/statements.sol",
"./resolc-compiler-tests/fixtures/solidity/simple/yul_semantic/switch.sol",
"./resolc-compiler-tests/fixtures/solidity/translated_semantic_tests/errors/errors_by_parameter_type/test.json",
"./resolc-compiler-tests/fixtures/solidity/translated_semantic_tests/functionCall/return_size_bigger_than_expected/test.json",
"./resolc-compiler-tests/fixtures/solidity/translated_semantic_tests/functionCall/return_size_shorter_than_expected_evm_version_after_homestead/test.json",
"./resolc-compiler-tests/fixtures/solidity/translated_semantic_tests/functionTypes/call_to_zero_initialized_function_type_ir/test.json",
"./resolc-compiler-tests/fixtures/solidity/translated_semantic_tests/saltedCreate/prediction_example/test.json",
"./resolc-compiler-tests/fixtures/solidity/translated_semantic_tests/viaYul/unary_operations/test.json"
]
}
EOF
RUST_LOG="info" ./retester \
--corpus "corp.json" \
--workdir "workdir" \
--number-of-nodes 5 \
--kitchensink "./substrate-node" \
--revive-dev-node "./revive-dev-node" \
--eth_proxy "./eth-rpc" \
> logs.log \
2> output.log
)