Wallet handle txs with conflicting account nonces #1864

azarovh · 2025-01-15T14:03:53Z

Unconfirmed txs in the wallet which uses outdated nonce are marked as conflicted now if confirmed tx is encountered on mainchain and the state of OutputCache is updated accordingly.

The behaviour is similar to abandoning tx.

Besides tests I had a wallet with conflicting orders which I couldn't open previously because of InconsistentOrderDuplicateNonce which is now resolved.

wallet/src/account/output_cache/mod.rs

wallet/src/wallet/tests.rs

wallet/src/account/output_cache/mod.rs

OBorce · 2025-01-21T17:15:51Z

wallet/src/account/output_cache/mod.rs


-                    conflicting_txs.push(unconfirmed);
+                if let Some((confirmed_account, confirmed_account_nonce)) = confirmed_account_nonce


this can probably be an else if, as the transactions that use a token id will be a superset of the transactions that use the token id and have a specific nonce.

I think that it'd make the code more confusing especially when modifying in the future. I replaced the vec with set to avoid duplicates.

wallet/src/account/output_cache/mod.rs

wallet/src/wallet/tests.rs

wallet/src/account/output_cache/mod.rs

azarovh · 2025-01-30T11:10:46Z

Fixed the problem when abandoning a tx that is already marked as conflicted leads to invalid state of internal fields of OutputCache (like account nonces).

wallet/src/wallet/tests.rs

wallet/src/account/output_cache/mod.rs

ImplOfAnImpl · 2025-01-31T07:55:48Z

wallet/src/account/output_cache/mod.rs

+                            }
+                        },
+                    },
+                    Entry::Vacant(_) => Err(WalletError::CannotFindTransactionWithId(tx_id)),


Is this also an invariant? If so, it deserves its own error.

But I also see that remove_tx removes the tx from txs unconditionally, possibly leaving it inside unconfirmed_descendants. If this situation is legitimate, we probably shouldn't return an error here.

To me it looks like an invariant. Because removing a tx removes its descendants as well.

ImplOfAnImpl · 2025-01-31T07:59:34Z

wallet/src/account/output_cache/mod.rs

+                            TxState::Abandoned
+                            | TxState::Confirmed(..)
+                            | TxState::InMempool(..)
+                            | TxState::Conflicted(..) => {
+                                Err(WalletError::CannotChangeTransactionState(
+                                    *tx.state(),
+                                    TxState::Conflicted(block_id),
+                                ))
+                            }


This doesn't look nice. This whole function is called during block processing, so if this situation happens, the wallet will stop working, right?

Abandoned, Confirmed and Conflicted probably can't happen here and are invariant violations, am I right? If so, a separate error would be nice.
But what about InMempool? It sounds like it should be possible. If so, we must handle it somehow. Would it be wrong to change the state from InMempool to Conflicted?

Looks like InMempool -> Conflicted is theoretically possible. Changed the logic to mark it as conflicted.

wallet/src/account/output_cache/mod.rs

wallet/src/wallet/tests.rs

OBorce · 2025-03-06T11:04:18Z

wallet/src/account/output_cache/mod.rs

+        for conflicting_tx in conflicting_txs {
+            let txs_to_rollback = self.remove_from_unconfirmed_descendants(conflicting_tx);
+
+            // Mark conflicting tx and its descendants as Conflicting and update OutputCache data accordingly
+            for tx_id in txs_to_rollback.iter().rev().copied() {


There is a problem that currently I think cannot happen but can happen in the future when we add TXs from Mempool.

The problem is with the rollback_tx_data, as the TXs are iterated in random order by tx_id, in a situation like:
Having unconfirmed txs for a delegation or token:
A(nonce 1) -> B(2) -> C(3) -> D(4)
If for some reason the conflict happens in the middle of the chain e.g. nonce 2 (currently it should always happen at the start of the chain from confirmed blocks, unless I am mistaken)
So if it happens at 2, all 2, 3, and 4 will be inside conflicting_txs, but not sorted by tx_id. So the rollback can happen to finish at 3 or 4, which will be unable to find last_parent, the tx A as a still unconfirmed tx for this delegation or token.

Thinking about it maybe you can have filter them out in n * log(d).
something like:
conflicting_descendents = BTreeSet::new();
for tx in conflicting_txs {
conflicting_descendents += find_descendents(tx);
}
conflicting_txs = conflicting_txs.filter(|tx| conflicting_descendetns.contains(tx))

This should also fix the double Conflict -> Conflict state

Am I getting it right that you suggest to filter out conflicting txs to be able to rollback descendants first and then conflicting txs themself?

txs_to_rollback is already sorted with the parent first and all descendants after it, and the loop is in reverse.
only the conflicting_txs are not sorted, so if we filter them out before this loop, everything will be ok and there will be no duplicate txs and no Conflict -> Conflict state.

So I checked the collection and I don't think filtering out parents is necessary as a result of remove_from_unconfirmed_descendants function call doesn't contain any duplicates anymore.

wallet/src/account/output_cache/mod.rs

ImplOfAnImpl · 2025-03-20T14:58:39Z

wallet/src/account/output_cache/mod.rs

-        let mut to_abandon = BTreeSet::from_iter([OutPointSourceId::from(tx_id)]);
+    ) -> Vec<Id<Transaction>> {
+        let mut all_txs = Vec::new();
+        let mut to_update = BTreeSet::from_iter([OutPointSourceId::from(tx_id)]);


I've just noticed that it's actually a set and not a vec. Why?
This can make descendants appear before parents in the result.

E.g. consider the following dependency graph:

/-->B-->\ A D \-->C-->/

We call remove_from_unconfirmed_descendants for A, so B and C are inserted into to_update. Let's assume Id(D) < Id(B) < Id(C), then on the next step B will be handled and D will be inserted into to_update before C. The resulting order will be "A, B, D, C, D" and this is what will be written to all_txs.

Using a vec would change it to "A, B, C, D, D".

But I guess returning duplicates is also not a good idea, so probably there should be an additional set, e.g. all_txs_as_set, that will be used to check whether a tx id already exists in all_txs.

A test that reproduces this would be nice to have.

Fixed, test added

ImplOfAnImpl · 2025-03-20T15:38:21Z

wallet/src/account/output_cache/mod.rs

+        for conflicting_tx in conflicting_txs {
+            let txs_to_rollback = self.remove_from_unconfirmed_descendants(conflicting_tx);
+


I think there are still some problems here:

As Boris has said, the txs in conflicting_txs are in a random order and descendants may come before parents.

remove_from_unconfirmed_descendants unconditionally puts ids from to_update into the result and to_update is initialized with the provided tx. So here txs_to_rollback may contain ids of trasansactions already rolled back on the previous iteration of the loop. Which will lead to CannotMarkTxAsConflictedIfInState(TxState::Conflicted).

But I'm not sure why we would need transactions from the mempool to reproduce this (which Boris also mentioned), it seems like it can happen anyway.
If so, a test that reproduces this problem would be nice to have.

I believe this is fixed in the previous comment with diamond deps

ImplOfAnImpl · 2025-03-20T16:03:47Z

wallet/src/account/output_cache/mod.rs

+            }
+        }
+
+        for output in tx.outputs() {


Could be nitpicking, but I'd suggest rolling back stuff in the exact opposit order of how it was added. I.e. first roll back outputs (in the reverse order too) and then inputs.
Currently it doesn't matter of course, but maybe our chain will evolve in the future and make the order matter.

I changed the order to rev(), but processing outputs before inputs break logic that searches for utxo parent, so I'd leave it this way

wallet/src/account/mod.rs

wallet/src/wallet/tests.rs

ImplOfAnImpl · 2025-03-25T11:37:44Z

wallet/src/account/output_cache/mod.rs

    }

-    fn uses_token(&self, unconfirmed_tx: &WalletTx, frozen_token_id: &TokenId) -> bool {
+    fn violates_frozen_token(&self, unconfirmed_tx: &WalletTx, frozen_token_id: &TokenId) -> bool {


Btw, probably this should check tx outputs too?

Not sure that it's possible to have tokens in the outputs without inputs

wallet/src/account/output_cache/mod.rs

azarovh requested review from erubboli, ImplOfAnImpl and OBorce as code owners January 15, 2025 14:03

azarovh marked this pull request as draft January 15, 2025 14:33

OBorce reviewed Jan 15, 2025

View reviewed changes

wallet/src/account/output_cache/mod.rs Outdated Show resolved Hide resolved

wallet/src/wallet/tests.rs Show resolved Hide resolved

azarovh force-pushed the fix/wallet_remove_conflicting_txs branch 2 times, most recently from 3a2fd08 to 6c83ac1 Compare January 21, 2025 09:11

azarovh marked this pull request as ready for review January 21, 2025 09:17

azarovh requested a review from OBorce January 21, 2025 09:17

OBorce approved these changes Jan 21, 2025

View reviewed changes

ImplOfAnImpl approved these changes Jan 27, 2025

View reviewed changes

azarovh force-pushed the fix/wallet_remove_conflicting_txs branch from 8be3bb5 to 3327b22 Compare January 29, 2025 09:56

ImplOfAnImpl approved these changes Jan 31, 2025

View reviewed changes

azarovh force-pushed the fix/wallet_remove_conflicting_txs branch from caa2c45 to f87218f Compare March 5, 2025 14:10

azarovh mentioned this pull request Mar 6, 2025

Withdraw from Delegation ERROR on node-gui #1889

Closed

OBorce reviewed Mar 6, 2025

View reviewed changes

ImplOfAnImpl reviewed Mar 6, 2025

View reviewed changes

wallet/src/account/output_cache/mod.rs Outdated Show resolved Hide resolved

ImplOfAnImpl requested changes Mar 7, 2025

View reviewed changes

azarovh force-pushed the fix/wallet_remove_conflicting_txs branch 2 times, most recently from 13bf848 to c1e3856 Compare March 17, 2025 11:02

azarovh requested review from ImplOfAnImpl and OBorce March 17, 2025 11:02

azarovh force-pushed the fix/wallet_remove_conflicting_txs branch from c1e3856 to ca1a378 Compare March 17, 2025 13:12

OBorce reviewed Mar 18, 2025

View reviewed changes

wallet/src/account/output_cache/mod.rs Show resolved Hide resolved

OBorce approved these changes Mar 18, 2025

View reviewed changes

ImplOfAnImpl reviewed Mar 20, 2025

View reviewed changes

ImplOfAnImpl reviewed Mar 25, 2025

View reviewed changes

azarovh force-pushed the fix/wallet_remove_conflicting_txs branch 2 times, most recently from 3528e2c to 34b383f Compare April 8, 2025 10:40

azarovh force-pushed the fix/wallet_remove_conflicting_txs branch from 34b383f to 9722bd7 Compare April 8, 2025 10:45

azarovh requested a review from ImplOfAnImpl April 8, 2025 11:15

ImplOfAnImpl approved these changes Apr 14, 2025

View reviewed changes

azarovh force-pushed the fix/wallet_remove_conflicting_txs branch from 9722bd7 to e8db935 Compare April 14, 2025 16:29

azarovh added 13 commits April 15, 2025 11:08

Remove descendants of conflicting txs

0e60d72

Use ref for AccountType::From

ba6de53

Check nonces while updating conflicting txs

5c985d5

Confirmed tx should not be marked as conflicted

e9b1554

Fix comments

4b875f5

Error on changing to same tx state

1fd0933

Fix abandoning conflicting txs

3693575

Fix comments

e85facc

Fix conflicting duplicates

0b32c17

Fix comments

2b903d4

Handle delegations, orders in rollback

f6fc9c3

Fix comments

342276f

Fix after rebase

be11da1

azarovh force-pushed the fix/wallet_remove_conflicting_txs branch from e8db935 to be11da1 Compare April 15, 2025 08:09

azarovh changed the base branch from master to fix/generate_transaction_graph_fee April 15, 2025 08:09

Base automatically changed from fix/generate_transaction_graph_fee to master April 15, 2025 11:16

azarovh merged commit 2729c09 into master Apr 15, 2025
18 checks passed

azarovh deleted the fix/wallet_remove_conflicting_txs branch April 15, 2025 11:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wallet handle txs with conflicting account nonces #1864

Wallet handle txs with conflicting account nonces #1864

azarovh commented Jan 15, 2025

OBorce Jan 21, 2025

azarovh Jan 22, 2025

azarovh commented Jan 30, 2025

ImplOfAnImpl Jan 31, 2025

azarovh Feb 3, 2025

ImplOfAnImpl Jan 31, 2025

azarovh Feb 3, 2025 •

edited

Loading

OBorce Mar 6, 2025

OBorce Mar 6, 2025

azarovh Mar 11, 2025 •

edited

Loading

OBorce Mar 13, 2025

azarovh Mar 17, 2025

ImplOfAnImpl Mar 20, 2025

azarovh Apr 8, 2025

ImplOfAnImpl Mar 20, 2025

azarovh Apr 8, 2025

ImplOfAnImpl Mar 20, 2025

azarovh Apr 8, 2025

ImplOfAnImpl Mar 25, 2025

azarovh Apr 8, 2025


		conflicting_txs.push(unconfirmed);
		if let Some((confirmed_account, confirmed_account_nonce)) = confirmed_account_nonce

		for conflicting_tx in conflicting_txs {
		let txs_to_rollback = self.remove_from_unconfirmed_descendants(conflicting_tx);

Wallet handle txs with conflicting account nonces #1864

Wallet handle txs with conflicting account nonces #1864

Conversation

azarovh commented Jan 15, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

azarovh commented Jan 30, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

azarovh Feb 3, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

azarovh Mar 11, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

azarovh Feb 3, 2025 •

edited

Loading

azarovh Mar 11, 2025 •

edited

Loading