3.0 Candidate: Support for Any Number of Descriptors

[thunderbiscuit]

Overview

This issue is the starting point for discussion on the idea of adding support for any number of keychains in the Wallet type in a future major/breaking release.

I have an PR (#226) digging into what such an API could look like, but it's really not the only way to do it and I invite others to also poke into this if they're interested.

Pros and Cons

✅ The wallet does away with the idea of main/change (external/internal) descriptors, which are fairly narrow-focused and less generally-applicable to all situations and requirements.
✅ Adding to the above: our wallet methods are currently sometimes not explicit enough, and produce unexpected results (for example calling Wallet::peek_address with KeychainKind::Internal will actually return an address on your External keychain if your wallet doesn't have an Internal keychain).
✅ Feature-parity for the Wallet type with application software in the industry that offers this feature. Examples: Blockstream Green, Sparrow Wallet, Blue Wallet, Samourai. All of those can provide a unified balance for the user over a number of descriptors/keychains.
❌ This potentially adds complexity to all wallet methods that deal with the keychains, because users need to handle the multiple keychains more explicitly. I'm not actually sure this is necessarily the case for all methods and under all API designs, just pointing it out here. More keychains = maybe more complexity at the call sites.
❌ Because the workflow of external/internal descriptors is embedded deep into the current Wallet API, the rewrite and review cost for shipping this is likely fairly high. The good thing is (at least so far in my exploration in #226), the reach of the changes is really limited to the wallet module. The separation with bdk_chain structures is really well done, and the inner logic is really built to handle this use case and so work out of the box really well (as far as I can tell). The real changes are mostly all at the Wallet and user-facing API level (not so much at the domain-logic level).

Some questions that still need answering

1. Who needs this? How would they use it?
2. New changesets and persistence. How problematic is that?

[evanlinjin]

The main concern I have is with the changeset design — this might be partly my oversight. I originally designed bdk_wallet::ChangeSet under the assumption that we wouldn't introduce flexible descriptor counts later on.

Some questions on my mind:

• For users who need this flexible descriptor count feature, do they require backward compatibility with the existing 2-descriptor-only Wallet? If not, maybe we can consider introducing another structure.
• Since we're introducing more flexibility, should we also consider use cases where the wallet isn't descriptor-based — e.g., silent payments?

[thunderbiscuit]

Yes the question of whether support for this feature is worth the Changeset type breaks (or break of the backwards-compatibility) across major releases is a good one, and/or whether we could/should look at introducing another structure (I need to understand more what you mean by that though, particularly around how much technical debt it introduces and the value to users of a clean break vs bolting new pieces on top of what we have). I personally think it sets us up nicely moving forward (including for things like your second point) so worth the time/effort, but I do think it'd be great to lay out the production use cases for this before going full steam ahead.

As for extra flexibility, I feel like that might be useful and I'm curious/optimistic we might be able to make this work with the idea of the wallet KeyRing laid out in #226 (a pre-Wallet structure that builds, holds, and controls the descriptors and keys and related data necessary for the Wallet to function). If so that would tie in well with the multi-descriptor wallet idea and in fact become indeed even more general.