upgrade(swap): Concurrent syncing, bdk upgrade, refactors (#180)

* upgrade sqlx to 0.8, add bdk_wallet and bdk_electrum

The new dependencies are part of the bdk upgrade and
include the improved wallet code.
They, too, depend on sqlite3.
However, they use a newer version than we currently use via sqlx.
This necessitated the sqlx upgrade.
This entailed trivial changes (use Pool directly instead of pool.acquire()).
We might have to fix the CI as well, I kept getting compile
errors from the macro until I ran swap/sqlx_dev_setup.sh.

* move old wallet code to extra module

* fix fee estimation for old client

* bump bitcoin crate, add new wallet constructor

* remove unused old Client, move code around for better readibility

* make Wallet generic over Persister (database) and move more code around for readibility

* add script history, start reimplementing client methods

* update some imports

* cargo fmt

* Add comments, fix fee estimation, address generation and status_of_script

* redo state update and wallet sync

* fix bitcoin address validation and more imports, use Amount everywhere

* fix tx cancel, lock, punish, redeem, refund

* fix bitcoin::Address de-/serialisation

* fix more address validation

* fix more address parsing and validation, also some more imports

* cargo fmt

* fix wallet initialization, start wallet migration

* fail test instead of ignoring it

* perform full scan on creation, load from db if it exists

* add more wallet info, fix wallet initialization

* fix: default to null in config

* migrate from old wallet if needed

* change something

* fix some tests

* temporarily patch bdk_wallet and bdk_electrum

* fix more tests

* fix missing rustls

* asb: only start tor client if register_hidden_service=true in the config

* fix: use p2wsh_signature_hash instead of p2wpkh_signature_hash

* fix some bitcoin address parsing and fee rate parsing

* dprint fmt

* add bitcoin-harness to this project and update to the new bitcoin version

* fix max_givible again

* create electrum client separately from wallet, clean up some code

* add comment

* ignore .env.development

* log config file path on ./asb config

* feat(monero-sys): Initial commit. Regtest integration test. Wrapper around basic Wallet functions, depends on monero#9464

* Revert "feat(monero-sys): Initial commit. Regtest integration test. Wrapper around basic Wallet functions, depends on monero#9464"

This reverts commit 14a5b4c.

* upgrade to rust toolchain 1.81

* Use new bdk update for code from master

* fix

* remove

* fix: add empty .gitmodules file to fix Docker build

* fix: clean up submodule references

* fix: properly declare monero submodule with ignore flag

* fix(wallet, bdk): only reveal new address if absolutely necessary

* fix: private keys not loaded into bdk wallet

* refactor: sync wallet progress log

* dprint fmt

* refactor: move bitcoin-harness to outside repo

* refactor: remove redundant log message

* Display sync progress

* Remove redundant arg to  swap/tests/harness/mod.rs function

* fix: call rustls::crypto::ring::default_provider()

* dprint fmt

* refactor: remove debug code

* refactor: move old bdk wallet export to own function, clear log messages

* remove old migr for testnets (checksum mismatch), remove balance and stringified last revealed addresses from migration export

* use revalidate_network function, remove redundant drop

* Display progress of background tasks, TauriBackgroundProgressHandle struct

* fix: almost satisfy clippy

* fix: gen-bindings error

* feat: add BackgroundRefund background type

* feat: use builder pattern for constructing Bitcoin wallet

* dprint ftm

* sync electrum in seperate thread

* do not allow user to start sync while sync is in progress

* remove redundant log message

* display random buffer in AlertWithLinearProgress progress

* fix: use TauriContextStatusEvent.Available), dont show syncing wallet spinner if not syncing

* differentiate between TestWalletBuilder and WalletBuilder

* satisfy clippy

* remove custom BackgroundRefund event, move into background process architecture

* refactor

* dprint fmt

* progress: get unit tests compiling

* fix: bitcoin unit tests specify const values like sync_interval

* fix: get unit tests passing

* make clippy happy

* feat: display full sync progress, fix unit test import issues

* dprint fmt

* make clippy happy, use u32 for target_block and not usize

* always spawn tor for asb

* refactor: remove gen_background_progress_id and just use Uuid::new_v4()

* refactor(hooks.ts): clarify comment on useConservativeBitcoinSyncProgress

* fix typo

* refactor: do not let WalletBuilder take entire env struct

* dprint fmt

* refactor: remove default feature from workspace patch of bdk

* first try for concurrent syncing

* refactor: concurrent syncing

* fix(wallet.rs): Safely convert FeeRate from btc / kb to sats / kwu

* feat(wallet.rs): persist published Bitcoin transactions without requiring re-scan

This allows us to compute an updated Bitcoin balance without requiring a re-scan

* refactor(wallet.rs): use just 5 concurrent sync requests

* refactor: display snackbar error when Wallet refresh fails

* fix: add missing space

* dprint fmt

* refactor: fancy traits for the CumulativeProgress struct, allow limiting amount of callback calls

* make clippy happy

* dprint fmt

* refactor: clearly differntiate between SyncMutex and TokioMutex, use traits for converting to Arc<Mutex<_>>, move sync_ext into own moid

* fix: skip syncing if no spks in wallet

* fix: update bdk.sh to test migration from old wallet (pre 1.0.0 bdk) to new bdk

* fix: increase bitcoin_lock_confirmed_timeout in RegTest env to 5 minutes

* refactor: avoid usize where possible, create persistence only after full scan, transmit assumed_total for full scan to tauri, add some icons to progress displays

* make clippy happy

* fix(ci): change rust toolchain 1.81

* fix(cross compilation arm): use ring instead of aws-lc-rs

* fmt

* ignore failing rendezvous tests

* fix printing_status_change_doesnt_spam_on_same_status

* fix: given_bitcoin_address_network_mismatch_then_error test

* ignore list_sellers_should_report_all_registered_asbs_with_a_quote test

* feat: add tor icon

* refactor(wallet.rs): reorder struct by abstraction level

* refactor(bitcoin wallet): chunk size for syncing

* fix(integration tests): decrease sync interval to 3s

* fix(integration tests): parse_rpc_err method to take new bdk error, not old one

* add changelog entry

---------

Co-authored-by: Binarybaron <[email protected]>
Co-authored-by: Mohan <[email protected]>

Author: Einliterflasche

.github/workflows/build-release-binaries.yml

@@ -66,7 +66,7 @@ jobs:
 
       - uses: dtolnay/rust-toolchain@master
         with:
-          toolchain: "1.80"
+          toolchain: "1.81"
 
       - name: Cross Build ${{ matrix.target }} ${{ matrix.bin }} binary
         if: matrix.target == 'armv7-unknown-linux-gnueabihf'

.github/workflows/ci.yml

@@ -19,7 +19,7 @@ jobs:
 
       - uses: dtolnay/rust-toolchain@master
         with:
-          toolchain: "1.80"
+          toolchain: "1.81"
           components: clippy,rustfmt
 
       - uses: Swatinem/[email protected]
@@ -127,7 +127,7 @@ jobs:
 
       - uses: dtolnay/rust-toolchain@master
         with:
-          toolchain: "1.80"
+          toolchain: "1.81"
           targets: armv7-unknown-linux-gnueabihf
 
       - name: Install dependencies required by Tauri v2 (ubuntu only)

.github/workflows/draft-new-release.yml

@@ -53,7 +53,7 @@ jobs:
         id: make-commit
         env:
           DPRINT_VERSION: "0.39.1"
-          RUST_TOOLCHAIN: "1.80"
+          RUST_TOOLCHAIN: "1.81"
         run: |
           rustup component add rustfmt --toolchain "$RUST_TOOLCHAIN-x86_64-unknown-linux-gnu"
           curl -fsSL https://dprint.dev/install.sh | sh -s $DPRINT_VERSION

.gitignore

@@ -1,2 +1,4 @@
 target
-.vscode
+.vscode
+.claude/settings.local.json
+.DS_Store

.gitmodules

@@ -0,0 +1,6 @@
+# Explicitly declare submodules that no longer exist
+# This prevents Docker build errors when trying to clone missing submodules
+[submodule "monero-sys/monero"]
+path = monero-sys/monero
+url = https://github.com/monero-project/monero.git
+ignore = all

CHANGELOG.md

@@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+- CLI + ASB + GUI: We upgraded dependencies related to the Bitcoin wallet. When you boot up the new version for the first time, a migration process will be run to convert the old wallet format to the new one. This might take a few minutes. We also fixed a bug where we would generate too many unused addresses in the Bitcoin wallet which would cause the wallet to take longer to start up as time goes on.
+- GUI: We display detailed progress about running background tasks (Tor bootstrapping, Bitcoin wallet sync progress, etc.)
+
 ## [1.0.0-rc.21] - 2025-05-15
 
 ## [1.0.0-rc.20] - 2025-05-14