Skip to content

fix: spks_of_all_keychains() shouldn't return an infinite iterator for non-wildcard descriptors#1093

Merged
evanlinjin merged 3 commits intobitcoindevkit:masterfrom
danielabrozzoni:fix/electrum_scan_bug
Sep 1, 2023
Merged

fix: spks_of_all_keychains() shouldn't return an infinite iterator for non-wildcard descriptors#1093
evanlinjin merged 3 commits intobitcoindevkit:masterfrom
danielabrozzoni:fix/electrum_scan_bug

Conversation

@danielabrozzoni
Copy link
Copy Markdown
Member

@danielabrozzoni danielabrozzoni commented Aug 24, 2023

Description

When you pass a non-wildcard descriptor in new_with_range, we make
sure that the range length is at most 1; if that's not the case, we
shorten it.
We would previously use new_with_range without this check and with a
non-wildcard descriptor in spks_of_all_keychains, this meant creating
a spkiterator that would go on producing the same spks over and over
again, causing some issues with syncing on electrum/esplora.

To reproduce the bug, run in example-crates/example_electrum:

cargo run "sh(wsh(or_d(c:pk_k(cPGudvRLDSgeV4hH9NUofLvYxYBSRjju3cpiXmBg9K8G9k1ikCMp),c:pk_k(cSBSBHRrzqSXFmrBhLkZMzQB9q4P9MnAq92v8d9a5UveBc9sLX32))))#zp9pcfs9" scan

Changelog notice

  • Fixed a bug where KeychainTxOutIndex::spks_of_all_keychains/KeychainTxOutIndex::spks_of_keychain would return an iterator yielding infinite spks even for non-wildcard descriptors.

Checklists

All Submissions:

  • I've signed all my commits
  • I followed the contribution guidelines
  • I ran cargo fmt and cargo clippy before committing

@danielabrozzoni danielabrozzoni changed the title fix: Use SpkIterator::new in spks_of{_all}_keychains fix: spks_of_all_keychains() shouldn't return an infinite iterator for non-wildcard descriptors Aug 24, 2023
@notmandatory notmandatory added this to the 1.0.0-alpha.2 milestone Aug 24, 2023
@danielabrozzoni danielabrozzoni mentioned this pull request Aug 24, 2023
4 tasks
Copy link
Copy Markdown
Member

@evanlinjin evanlinjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for spotting and fixing this @danielabrozzoni!

I think SpkIterator::new_with_range should also detect non-wildcard descriptors, and the implementation of SpkIterator::new can be changed to this:

    /// Creates a new script pubkey iterator starting at 0 from a descriptor.
    pub fn new(descriptor: D) -> Self {
        SpkIterator::new_with_range(descriptor, 0..=BIP32_MAX_INDEX)
    }

This means, SpkIterator:new_with_range will sometimes return empty iterators.

@LLFourn
Copy link
Copy Markdown
Collaborator

LLFourn commented Aug 25, 2023

+1 for what @evanlinjin suggested above.

@danielabrozzoni
Copy link
Copy Markdown
Member Author

danielabrozzoni commented Aug 25, 2023

I think SpkIterator::new_with_range should also detect non-wildcard descriptors, and the implementation of SpkIterator::new can be changed to this:

I tried to implement this by setting end = next_index in new_with_range for non-wildcard descriptors, but I'm seeing some tests failing, investigating now...

..into account

When you pass a non-wildcard descriptor in `new_with_range`, we make
sure that the range length is at most 1; if that's not the case, we
shorten it.
We would previously use `new_with_range` without this check and with a
non-wildcard descriptor in `spks_of_all_keychains`, this meant creating
a spkiterator that would go on producing the same spks over and over
again, causing some issues with syncing on electrum/esplora.

To reproduce the bug, run in `example-crates/example_electrum`:
```
cargo run "sh(wsh(or_d(c:pk_k(cPGudvRLDSgeV4hH9NUofLvYxYBSRjju3cpiXmBg9K8G9k1ikCMp),c:pk_k(cSBSBHRrzqSXFmrBhLkZMzQB9q4P9MnAq92v8d9a5UveBc9sLX32))))#zp9pcfs9" scan
```
@danielabrozzoni
Copy link
Copy Markdown
Member Author

I tried to implement this by setting end = next_index in new_with_range for non-wildcard descriptors, but I'm seeing some tests failing, investigating now...

Turns out that this was wrong. I'm now shortening whichever range passed to new_with_range to have length at most 1 if the descriptor doesn't have a wildcard, and it works perfectly. Lmk what you think :)

Copy link
Copy Markdown
Member

@evanlinjin evanlinjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this. This was my oversight.

As discussed (via DM), it'll be better to have non-wildcard descriptors only returns something on derivation index 0.

The follow tests should pass:

// non index-0 should NOT return an spk
assert_eq!(SpkIterator::new_with_range(&no_wildcard_descriptor, 1..1).next(), None);
assert_eq!(SpkIterator::new_with_range(&no_wildcard_descriptor, 1..=1).next(), None);
assert_eq!(SpkIterator::new_with_range(&no_wildcard_descriptor, 1..2).next(), None);
assert_eq!(SpkIterator::new_with_range(&no_wildcard_descriptor, 1..=2).next(), None);

Copy link
Copy Markdown
Member

@evanlinjin evanlinjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK e48b911

Tested with example_electrum with non-wildcard descriptor.
Left a non-critical nit.

@evanlinjin evanlinjin merged commit 2867e88 into bitcoindevkit:master Sep 1, 2023
@danielabrozzoni danielabrozzoni deleted the fix/electrum_scan_bug branch October 11, 2023 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

4 participants