apollo_gateway: extract async mempool query from blocking task

[itamar-starkware]

TL;DR

Optimize transaction validation by prefetching mempool data asynchronously instead of blocking on async operations.

What changed?

• Refactored ProcessTxBlockingTask to receive the mempool check result instead of performing it during execution
• Modified the Gateway to prefetch the mempool result in an async context before creating the blocking task
• Updated extract_state_nonce_and_run_validations to accept the prefetched mempool result instead of performing the check itself
• Removed the runtime handle from the blocking task as it's no longer needed
• Updated all related tests to accommodate the new approach

How to test?

• Run existing tests to ensure they pass with the new implementation
• Verify that transaction processing still works correctly with the prefetched mempool data
• Check that the gateway correctly handles both positive and negative mempool check results

Why make this change?

This change eliminates a performance bottleneck where a blocking thread was calling block_on for an async operation. By prefetching the mempool data in an async context before creating the blocking task, we avoid blocking-on-async in a blocking thread, which improves overall performance and resource utilization. This is particularly important for the gateway component which handles transaction processing and needs to maintain high throughput.

[itamar-starkware]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• apollo_gateway: test account tx in mempool influence on validation #9950
• apollo_gateway: extract async mempool query from blocking task #9941 👈 (View in Graphite)
• apollo_gateway: use get_nonce outside extract_state_nonce_and_run_validations #10009
• apollo_gateway: add get_nonce function to StatefulValidator trait #10008
• apollo_gateway: insert stateful_tx_validator creating to ProcessTxBlockingTask constructor #10007
• apollo_gateway: split instantiate_validator #10123
• apollo_gateway: move instantiate_validator to be async #10122 : 1 other dependent PR (#10486 )
• apollo_gateway: move get_latest_block_info to be async #10121
• apollo_gateway: move get_block_info to be async #10120
• apollo_gateway: move get_state_reader_from_latest_block to be async #10119
• main-v0.14.1

This stack of pull requests is managed by Graphite. Learn more about stacking.

[reviewable-StarkWare]

This change is 

[ArniStarkware]

crates/apollo_gateway/src/gateway_test.rs line 333 at r2 (raw file):

        executable_tx: executable_invoke_tx(invoke_args()),
        is_account_tx_in_mempool,
    }

Can we simplify this test util?

Suggestion:

fn process_tx_task(
    stateful_transaction_validator_factory: MockStatefulTransactionValidatorFactoryTrait,
) -> ProcessTxBlockingTask {
    let is_account_tx_in_mempool = false;
    ProcessTxBlockingTask {
        stateful_tx_validator_factory: Arc::new(stateful_transaction_validator_factory),
        state_reader_factory: Arc::new(MockStateReaderFactory::new()),
        executable_tx: executable_invoke_tx(invoke_args()),
        is_account_tx_in_mempool,
    }

[ArniStarkware]

Reviewable status: 0 of 4 files reviewed, 3 unresolved discussions (waiting on @itamar-starkware and @TzahiTaub)

---

crates/apollo_gateway/src/gateway.rs line 162 at r2 (raw file):

            .mempool_client
            .account_tx_in_pool_or_recent_block(executable_tx.contract_address())
            .await;

We don't want to call the mempool for every transaction. That is very cost-ineffective:

Suggestion:

        // Please clean up this code - no need for this value to be mut.
        let mut is_account_tx_in_mempool = None;
        if tx.nonce() == Nonce(Felt::ONE) && account_nonce == Nonce(Felt::ZERO) {
            // Value is Some only if the question `is_account_tx_in_mempool` is in any way interesting.
            // Otherwise we don't care - it is irrelevant for skip validate.
        let is_account_tx_in_mempool: Optional<bool> = Some(self
            .mempool_client
            .account_tx_in_pool_or_recent_block(executable_tx.contract_address())
            .await.map_err(..).inspect(..)?);
        }

---

crates/apollo_gateway/src/stateful_transaction_validator.rs line 309 at r2 (raw file):

    }

    Ok(false)

Philosophically, I would extract this error handling outside of the blocking task and next to the query to the mempool.

Suggestion:

fn skip_stateful_validations(
    tx: &ExecutableTransaction,
    account_nonce: Nonce,
    is_account_tx_in_mempool: bool,
) -> StatefulTransactionValidatorResult<bool> {
    if let ExecutableTransaction::Invoke(ExecutableInvokeTransaction { tx, .. }) = tx {
        // check if the transaction nonce is 1, meaning it is post deploy_account, and the
        // account nonce is zero, meaning the account was not deployed yet.
        if tx.nonce() == Nonce(Felt::ONE) && account_nonce == Nonce(Felt::ZERO) {
            let account_address = tx.sender_address();
            debug!("Checking if deploy_account transaction exists for account {account_address}.");
            // We verify that a deploy_account transaction exists for this account. It is sufficient
            // to check if the account exists in the mempool since it means that either it has a
            // deploy_account transaction or transactions with future nonces that passed
            // validations.
            return Ok(is_account_tx_in_mempool);
        }
    }

    Ok(false)

[ArniStarkware]

Reviewable status: 0 of 4 files reviewed, 3 unresolved discussions (waiting on @itamar-starkware and @TzahiTaub)

---

crates/apollo_gateway/src/gateway.rs line 162 at r2 (raw file):

Previously, ArniStarkware (Arnon Hod) wrote…

We don't want to call the mempool for every transaction. That is very cost-ineffective:

another thing to make sure before we even start the call to the mempool - that the transaction in question is an invoke transaction.

I.E. move all the logic related to which transaction we query on.