Use BDK events in `update_payment_store` #658

Camillarhi · 2025-10-09T16:20:13Z

This PR updates update_payment_store to use BDK 2.2’s WalletEvent stream during sync instead of iterating over the full list of wallet transactions every time. The new event-based approach reduces redundant work and ensures the payment store stays in sync with only the changes that actually occurred.

It also sets up the foundation for RBF support in #628 with WalletEvent::TxReplaced. Since #628 depends on this event handling, this PR should be merged first.

This PR will also address #452

ldk-reviews-bot · 2025-10-09T16:20:16Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

ldk-reviews-bot · 2025-10-18T00:01:09Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-10-20T00:01:43Z

🔔 2nd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-10-22T00:02:22Z

🔔 3rd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-10-25T00:00:15Z

🔔 4th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-10-27T00:00:59Z

🔔 5th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-10-29T00:01:52Z

🔔 6th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

Excuse the delay here!

Unfortunately I don't think we can make the move until we get corresponding functionality for all chain sources, i.e., also for bitcoind/apply_block. Will raise that with the BDK folks to make some progress.

I now opened bitcoindevkit/bdk_wallet#336 to add the missing APIs we need. In the meantime we can see to get this as close to being mergeable as possible.

tnull · 2025-10-29T11:31:24Z

src/wallet/mod.rs

 				})?;

-				self.update_payment_store(&mut *locked_wallet).map_err(|e| {
+				let events_vec: Vec<WalletEvent> = events.into_iter().collect();


I don't think this re-allocation is necessary?

Thanks! I have removed the re-allocation

tnull · 2025-10-29T11:35:00Z

src/wallet/mod.rs

 			);
 		}

 		match locked_wallet.apply_block(block, height) {


Ugh, seems there is no corresponding apply_block_events method. I think we need that before actually moving forward here. Will raise it with the BDK folks.

I saw that the PR on BDK wallet has been merged and added to the next release milestone. This will be updated as soon as there is a new release on BDK wallet

tnull · 2025-10-29T11:37:20Z

src/wallet/mod.rs

+		}
+
+		self.payment_store
+			.list_filter(|p| {


Hmm, ugh, that's already slow right now, but will be prohibitively slow when we don't keep our entire payment store in-memory. I think we can't get around adding another persisted lookup table that tracks RBF-Txid to original-Txid.

Yes, this is true, if we have another table, the lookup will be faster and the original Txid can be updated when the RBF-Txid for example, has a confirmed event from BDK

@tnull, I created a new persisted ReplacedTransactionStore that maintains lookups from any txid in an RBF chain to its associated payment. Store entries are automatically cleaned up when any transaction in the chain confirms. This keeps lookups fast even with large payment histories.

@tnull, I created a new persisted ReplacedTransactionStore that maintains lookups from any txid in an RBF chain to its associated payment. Store entries are automatically cleaned up when any transaction in the chain confirms. This keeps lookups fast even with large payment histories.

Huh, why do we need a whole other module/store for this? Let's just use a HashMap<Txid, Txid> and be done with it? Or do we need all that additionally tracked data somehow?

I guess we could use a DataStore implementation for this, but I don't quite see why we need to track ConfirmationStatus and latest_update_timestamp?

Huh, why do we need a whole other module/store for this? Let's just use a HashMap<Txid, Txid> and be done with it? Or do we need all that additionally tracked data somehow?

Yes, we need to track this data for faster lookup of replaced transaction IDs instead of a full iteration

I guess we could use a DataStore implementation for this, but I don't quite see why we need to track ConfirmationStatus and latest_update_timestamp?

You are right about the ConfirmationStatus and latest_update_timestamp, they were part of my original design, and as I proceeded with my implementation, I decided to clean the store instead upon confirmation of any of the transactions. I will go ahead and remove them. I also used a DataStore implementation for this.

ldk-reviews-bot · 2025-11-08T02:35:20Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-11-10T02:35:49Z

🔔 2nd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-11-12T02:36:22Z

🔔 3rd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-11-15T00:00:24Z

🔔 4th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-11-17T00:01:25Z

🔔 5th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-11-19T00:02:36Z

🔔 6th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-11-22T00:00:13Z

🔔 7th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-11-24T00:01:01Z

🔔 8th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-11-26T00:01:25Z

🔔 9th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-11-29T00:00:52Z

🔔 10th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-12-01T00:01:37Z

🔔 11th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

@Camillarhi Mind adding a temporary commit in the beginning that changes the bdk_wallet dependency to include the changes of bitcoindevkit/bdk_wallet#336 ? Then we already can make use of them and get this PR in a state that is landable as soon as the changes are released (which could even happen this week).

Camillarhi · 2025-12-08T13:05:26Z

@Camillarhi Mind adding a temporary commit in the beginning that changes the bdk_wallet dependency to include the changes of bitcoindevkit/bdk_wallet#336 ? Then we already can make use of them and get this PR in a state that is landable as soon as the changes are released (which could even happen this week).

Sure, I'll make the update. I also confirmed the changes on BDK_wallet have been released

tnull

Thanks for updating! A few comments.

Generally, a lot of things are now happening at the same time (version bumps, code moves/prefactors, actual logic changes, etc). in just these two commits. It would be preferable if you could restructure the commit history to pull out prefactors and unrelated changes to individual commits, which would make following the actual logic changes a lot easier during review. Thanks!

tnull · 2025-12-09T09:04:10Z

Cargo.toml

 bdk_esplora = { version = "0.22.0", default-features = false, features = ["async-https-rustls", "tokio"]}
 bdk_electrum = { version = "0.23.0", default-features = false, features = ["use-rustls-ring"]}
-bdk_wallet = { version = "2.2.0", default-features = false, features = ["std", "keys-bip39"]}
+bdk_wallet = { version = "2.3.0", default-features = false, features = ["std", "keys-bip39"]}


Please make the BDK bump a dedicated commit.

Thanks, this has been updated

tnull · 2025-12-09T09:20:52Z

src/wallet/mod.rs


 	fn update_payment_store<'a>(
 		&self, locked_wallet: &'a mut PersistedWallet<KVStoreWalletPersister>,
+		events: &'a Vec<WalletEvent>,


If we use move semantics we can simplify the code quite a bit, and especially save on all the reallocations:

diff --git a/src/wallet/mod.rs b/src/wallet/mod.rs index 5982eabc..f65b3772 100644 --- a/src/wallet/mod.rs +++ b/src/wallet/mod.rs @@ -135,7 +135,7 @@ impl Wallet { Error::PersistenceFailed })?; - self.update_payment_store(&mut *locked_wallet, &events).map_err(|e| { + self.update_payment_store(&mut *locked_wallet, events).map_err(|e| { log_error!(self.logger, "Failed to update payment store: {}", e); Error::PersistenceFailed })?; @@ -167,15 +167,14 @@ impl Wallet { fn update_payment_store<'a>( &self, locked_wallet: &'a mut PersistedWallet<KVStoreWalletPersister>, - events: &'a Vec<WalletEvent>, + mut events: Vec<WalletEvent>, ) -> Result<(), Error> { if events.is_empty() { return Ok(()); } - let sorted_events: Vec<_> = if events.len() > 1 { - let mut events_vec: Vec<_> = events.iter().collect(); - events_vec.sort_by_key(|e| match e { + if events.len() > 1 { + events.sort_by_key(|e| match e { WalletEvent::TxReplaced { .. } => 0, WalletEvent::TxUnconfirmed { .. } => 1, WalletEvent::TxConfirmed { .. } => 2, @@ -183,12 +182,9 @@ impl Wallet { WalletEvent::TxDropped { .. } => 4, _ => 5, }); - events_vec - } else { - events.iter().collect() }; - for event in sorted_events { + for event in events { match event { WalletEvent::TxConfirmed { txid, tx, block_time, .. } => { let cur_height = locked_wallet.latest_checkpoint().height(); @@ -207,14 +203,14 @@ impl Wallet { }; let payment_id = self - .find_payment_by_txid(*txid) + .find_payment_by_txid(txid) .unwrap_or_else(|| PaymentId(txid.to_byte_array())); let payment = self.create_payment_from_tx( locked_wallet, - *txid, + txid, payment_id, - tx, + &tx, payment_status, confirmation_status, None, @@ -261,14 +257,14 @@ impl Wallet { }, WalletEvent::TxUnconfirmed { txid, tx, old_block_time: None } => { let payment_id = self - .find_payment_by_txid(*txid) + .find_payment_by_txid(txid) .unwrap_or_else(|| PaymentId(txid.to_byte_array())); let payment = self.create_payment_from_tx( locked_wallet, - *txid, + txid, payment_id, - tx, + &tx, PaymentStatus::Pending, ConfirmationStatus::Unconfirmed, None, @@ -277,7 +273,7 @@ impl Wallet { }, WalletEvent::TxReplaced { txid, conflicts, .. } => { let payment_id = self - .find_payment_by_txid(*txid) + .find_payment_by_txid(txid) .unwrap_or_else(|| PaymentId(txid.to_byte_array())); // Collect all conflict txids @@ -286,24 +282,21 @@ impl Wallet { for conflict_txid in conflict_txids { // Update the replaced transaction store - let replaced_tx_details = ReplacedOnchainTransactionDetails::new( - conflict_txid, - *txid, - payment_id, - ); + let replaced_tx_details = + ReplacedOnchainTransactionDetails::new(conflict_txid, txid, payment_id); self.replaced_tx_store.insert_or_update(replaced_tx_details)?; } }, WalletEvent::TxDropped { txid, tx } => { let payment_id = self - .find_payment_by_txid(*txid) + .find_payment_by_txid(txid) .unwrap_or_else(|| PaymentId(txid.to_byte_array())); let payment = self.create_payment_from_tx( locked_wallet, - *txid, + txid, payment_id, - tx, + &tx, PaymentStatus::Pending, ConfirmationStatus::Unconfirmed, None, @@ -999,7 +992,7 @@ impl Listen for Wallet { match locked_wallet.apply_block_events(block, height) { Ok(events) => { - if let Err(e) = self.update_payment_store(&mut *locked_wallet, &events) { + if let Err(e) = self.update_payment_store(&mut *locked_wallet, events) { log_error!(self.logger, "Failed to update payment store: {}", e); return; }

tnull · 2025-12-09T09:22:40Z

src/wallet/mod.rs

+
+		let sorted_events: Vec<_> = if events.len() > 1 {
+			let mut events_vec: Vec<_> = events.iter().collect();
+			events_vec.sort_by_key(|e| match e {


Why do we need to sort the events? Is it so we first process TxConfirmed before adjusting the height value in ChainTipChanged?

We process TXReplaced before TxUnconfirmed because when an RBF occurs, both a TXReplaced and a TxUnconfirmed event are triggered.
If TxUnconfirmed were processed first, it would create a new payment record for the unconfirmed transaction, which we don’t want. Instead, we only want the original transaction to have a payment record, and link the replaced transaction to it.

By handling TXReplaced first, we store the replaced transaction in ReplacedOnchainTransactionStore. Then, when the TxUnconfirmed event arrives later, it checks and avoids creating a duplicate payment record for the now-conflicting transaction.

As for TxConfirmed and ChainTipChanged, the order matters there too. If we processed ChainTipChanged (which updates the block height) before TxConfirmed, the confirmation might reference an incorrect or outdated height. Sorting ensures TxConfirmed uses the correct chain state.

tnull · 2025-12-09T09:26:39Z

src/wallet/mod.rs

+		payment_id: PaymentId, tx: &Transaction, payment_status: PaymentStatus,
+		confirmation_status: ConfirmationStatus, conflicting_txids: Option<Vec<Txid>>,
+	) -> PaymentDetails {
+		// TODO: It would be great to introduce additional variants for


Can we make this code move in a dedicated prefactor commit, so that git diff --color-moved clearly show the code move, before we then change anything in the following commits?

tnull · 2025-12-09T09:30:42Z

src/payment/replaced_transaction_store.rs

+	}
+}
+
+impl Writeable for ReplacedOnchainTransactionDetails {


Please use LDK's serialization macros (in this case impl_writeable_tlv_based) whenever you can.

tnull · 2025-12-09T09:35:16Z

src/wallet/mod.rs

+
+	fn find_payment_by_txid(&self, target_txid: Txid) -> Option<PaymentId> {
+		let direct_payment_id = PaymentId(target_txid.to_byte_array());
+		if self.payment_store.get(&direct_payment_id).is_some() {


I think this would unnecessarily clone the entry. Let's just add a contains implementation to DataStore (preferably in a separate prefactor commit).

Thanks, I have added a contains_key implementation in a separate commit.

tnull · 2025-12-09T09:43:51Z

src/wallet/mod.rs

+						.map(|p| p.new_txid)
+						.collect::<Vec<Txid>>();
+					for replaced_txid in replaced_txids {
+						self.replaced_tx_store.remove(&replaced_txid)?;


I think we can only remove the entry once we reach ANTI_REORG_DELAY, no? Otherwise we might see it confirmed, delete the entry, tip gets reorged, and we wouldn't track the original payment ID anymore?

Yes, thanks for pointing this out, this should be done when ANTI_REORG_DELAY is reached not on TxConfirmed event. I will make the update for it

…sactions Replace the full transaction list scan in `update_payment_store` with handling of BDK's `WalletEvent` stream during sync. This leverages the new events in BDK 2.2, reduces redundant work, and prepares the foundation for reliable RBF/CPFP tracking via `WalletEvent::TxReplaced`

Introduce a new lookup `ReplacedTransactionStore` that maps old/replaced transaction IDs to their current replacement transaction IDs, enabling reliable tracking of replaced transactions throughout the replacement chain. Key changes: - Add persisted storage for RBF replacement relationships - Link transactions in replacement trees using payment IDs - Remove entire replacement chains from persistence when any transaction in the tree is confirmed

Camillarhi · 2025-12-09T19:15:36Z

Thanks for updating! A few comments.

Generally, a lot of things are now happening at the same time (version bumps, code moves/prefactors, actual logic changes, etc). in just these two commits. It would be preferable if you could restructure the commit history to pull out prefactors and unrelated changes to individual commits, which would make following the actual logic changes a lot easier during review. Thanks!

Thanks, I've completed all the requested changes, including restructuring the commits to separate concerns.

ldk-reviews-bot · 2025-12-11T19:16:20Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull mentioned this pull request Oct 10, 2025

Fix Invalid Pending Payments and Strengthen RBF Tests #647

Open

Camillarhi force-pushed the payment-store-events-sync branch 9 times, most recently from 75ff700 to d3f7855 Compare October 15, 2025 20:32

Camillarhi marked this pull request as ready for review October 15, 2025 20:35

ldk-reviews-bot requested a review from tnull October 15, 2025 20:36

tnull reviewed Oct 29, 2025

View reviewed changes

Camillarhi force-pushed the payment-store-events-sync branch from d3f7855 to a2c8a55 Compare October 29, 2025 16:08

Camillarhi requested a review from tnull November 6, 2025 02:34

Camillarhi force-pushed the payment-store-events-sync branch 2 times, most recently from ecaae51 to 8609d97 Compare November 6, 2025 10:58

tnull reviewed Dec 2, 2025

View reviewed changes

Camillarhi force-pushed the payment-store-events-sync branch from 8609d97 to f6c2ccd Compare December 8, 2025 15:15

Camillarhi requested a review from tnull December 8, 2025 15:16

Camillarhi force-pushed the payment-store-events-sync branch 3 times, most recently from 2b60f1d to 7956129 Compare December 8, 2025 18:56

notmandatory mentioned this pull request Dec 8, 2025

Socratic Seminar #38 Topics NashBitDevs/nashbitdevs.org#114

Closed

tnull reviewed Dec 9, 2025

View reviewed changes

Camillarhi force-pushed the payment-store-events-sync branch 5 times, most recently from c6465cb to b442739 Compare December 9, 2025 17:33

Camillarhi added 5 commits December 9, 2025 20:04

refactor: Extract payment creation logic into create_payment_from_tx

0bd4afe

Bump BDK_Wallet to 2.3.0

0cab667

Add contains_key method to DataStore

9eb5584

Camillarhi force-pushed the payment-store-events-sync branch from b442739 to e13e503 Compare December 9, 2025 19:09

Camillarhi requested a review from tnull December 9, 2025 19:15

Use BDK events in update_payment_store #658

Are you sure you want to change the base?

Use BDK events in update_payment_store #658

Uh oh!

Conversation

Camillarhi commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ldk-reviews-bot commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ldk-reviews-bot commented Oct 18, 2025

Uh oh!

ldk-reviews-bot commented Oct 20, 2025

Uh oh!

ldk-reviews-bot commented Oct 22, 2025

Uh oh!

ldk-reviews-bot commented Oct 25, 2025

Uh oh!

ldk-reviews-bot commented Oct 27, 2025

Uh oh!

ldk-reviews-bot commented Oct 29, 2025

Uh oh!

tnull left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tnull Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ldk-reviews-bot commented Nov 8, 2025

Uh oh!

ldk-reviews-bot commented Nov 10, 2025

Uh oh!

ldk-reviews-bot commented Nov 12, 2025

Uh oh!

ldk-reviews-bot commented Nov 15, 2025

Uh oh!

ldk-reviews-bot commented Nov 17, 2025

Uh oh!

ldk-reviews-bot commented Nov 19, 2025

Uh oh!

ldk-reviews-bot commented Nov 22, 2025

Uh oh!

ldk-reviews-bot commented Nov 24, 2025

Uh oh!

ldk-reviews-bot commented Nov 26, 2025

Uh oh!

ldk-reviews-bot commented Nov 29, 2025

Uh oh!

ldk-reviews-bot commented Dec 1, 2025

Uh oh!

tnull left a comment

Choose a reason for hiding this comment

Uh oh!

Camillarhi commented Dec 8, 2025

Uh oh!

tnull left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Use BDK events in `update_payment_store` #658

Use BDK events in `update_payment_store` #658

Camillarhi commented Oct 9, 2025 •

edited

Loading

ldk-reviews-bot commented Oct 9, 2025 •

edited

Loading

tnull left a comment •

edited

Loading

tnull Nov 6, 2025 •

edited

Loading