Skip to content

Conversation

@darosior
Copy link
Member

@darosior darosior commented Oct 4, 2021

Writing unit tests for Miniscript descriptors i noticed that test/descriptor_tests's DoCheck() assumes that a descriptor would either contain only extended keys or only const pubkeys: if it detects an xpub in the descriptor it would assert the number of cached keys is equal to the number of keys in the descriptor, which does not hold if the descriptor also contains const (raw?) public keys since we only cache parent xpubs.

@fanquake fanquake added the Tests label Oct 4, 2021
@DrahtBot
Copy link
Contributor

DrahtBot commented Oct 4, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #22838 (descriptors: Be able to specify change and receiving in a single descriptor string by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@amadeuszpawlik
Copy link
Contributor

tACK 6ae2d37c7112656f3fb09e5fabc016345f73aa06, looks good to me

Copy link
Contributor

@rajarshimaitra rajarshimaitra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept + tACK 6ae2d37

Copy link
Contributor

@stratospher stratospher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested ACK 6ae2d37.

Checking mixed pubkeys is a nice addition to the test.

@Sjors
Copy link
Member

Sjors commented Dec 8, 2021

cc @achow101 @sipa

@achow101
Copy link
Member

achow101 commented Dec 8, 2021

While I like the idea of testing mixed ranged and const pubkeys, I don't think this is the right approach.

For starters, we have a flags field to indicate what to check for, so I think it would be better to add mixed_pubkeys as an additional flag rather than as a bool. Secondly, this doesn't really test that the right keys are there. Rather it just disables the cache checks when there are mixed ranged and const keys. IMO the correct thing to do would be to retain those checks, with modifications to allow for the mixed case when the appropriate flag is set.

@darosior
Copy link
Member Author

darosior commented Dec 8, 2021

Definitely agree the approach is clumsy. However this does test that we can parse descriptors with mixed const and extended pubkeys. Keeping the check on the number of xpubs cached in this case would require either adding a new getter to Descriptor or parsing the number of xpubs directly from the string, and both approaches sound overkill.

Pushed the change to use the flag instead of the boolean.

@achow101
Copy link
Member

achow101 commented Dec 8, 2021

diff --git a/src/test/descriptor_tests.cpp b/src/test/descriptor_tests.cpp
index 68584307b2..4d5ef381d0 100644
--- a/src/test/descriptor_tests.cpp
+++ b/src/test/descriptor_tests.cpp
@@ -74,6 +74,18 @@ std::string UseHInsteadOfApostrophe(const std::string& desc)
     return ret;
 }
 
+// Count the number of times the string "xpub" appears in a descriptor string
+static size_t CountXpubs(const std::string& desc)
+{
+    size_t count = 0;
+    size_t p = desc.find("xpub", 0);
+    while (p != std::string::npos) {
+        count++;
+        p = desc.find("xpub", p + 1);
+    }
+    return count;
+}
+
 const std::set<std::vector<uint32_t>> ONLY_EMPTY{{}};
 
 void DoCheck(const std::string& prv, const std::string& pub, const std::string& norm_prv, const std::string& norm_pub, int flags, const std::vector<std::vector<std::string>>& scripts, const std::optional<OutputType>& type, const std::set<std::vector<uint32_t>>& paths = ONLY_EMPTY,
@@ -172,7 +184,8 @@ void DoCheck(const std::string& prv, const std::string& pub, const std::string&
             // Check whether keys are in the cache
             const auto& der_xpub_cache = desc_cache.GetCachedDerivedExtPubKeys();
             const auto& parent_xpub_cache = desc_cache.GetCachedParentExtPubKeys();
-            if ((flags & RANGE) && !(flags & (DERIVE_HARDENED | MIXED_PUBKEYS))) {
+            const size_t num_xpubs = CountXpubs(pub1);
+            if ((flags & RANGE) && !(flags & (DERIVE_HARDENED))) {
                 // For ranged, unhardened derivation, None of the keys in origins should appear in the cache but the cache should have parent keys
                 // But we can derive one level from each of those parent keys and find them all
                 BOOST_CHECK(der_xpub_cache.empty());
@@ -184,13 +197,22 @@ void DoCheck(const std::string& prv, const std::string& pub, const std::string&
                     xpub.Derive(der, i);
                     pubkeys.insert(der.pubkey);
                 }
+                int count_pks = 0;
                 for (const auto& origin_pair : script_provider_cached.origins) {
                     const CPubKey& pk = origin_pair.second.first;
-                    BOOST_CHECK(pubkeys.count(pk) > 0);
+                    count_pks += pubkeys.count(pk);
                 }
-            } else if (!(flags & MIXED_PUBKEYS) && pub1.find("xpub") != std::string::npos) {
+                if (flags & MIXED_PUBKEYS) {
+                    BOOST_CHECK_EQUAL(num_xpubs, count_pks);
+                } else {
+                    BOOST_CHECK_EQUAL(script_provider_cached.origins.size(), count_pks);
+                }
+            } else if (num_xpubs > 0) {
                 // For ranged, hardened derivation, or not ranged, but has an xpub, all of the keys should appear in the cache
-                BOOST_CHECK(der_xpub_cache.size() + parent_xpub_cache.size() == script_provider_cached.origins.size());
+                BOOST_CHECK(der_xpub_cache.size() + parent_xpub_cache.size() == num_xpubs);
+                if (!(flags & MIXED_PUBKEYS)) {
+                    BOOST_CHECK(num_xpubs == script_provider_cached.origins.size());
+                }
                 // Get all of the derived pubkeys
                 std::set<CPubKey> pubkeys;
                 for (const auto& xpub_map_pair : der_xpub_cache) {
@@ -207,9 +229,15 @@ void DoCheck(const std::string& prv, const std::string& pub, const std::string&
                     xpub.Derive(der, i);
                     pubkeys.insert(der.pubkey);
                 }
+                int count_pks = 0;
                 for (const auto& origin_pair : script_provider_cached.origins) {
                     const CPubKey& pk = origin_pair.second.first;
-                    BOOST_CHECK(pubkeys.count(pk) > 0);
+                    count_pks += pubkeys.count(pk);
+                }
+                if (flags & MIXED_PUBKEYS) {
+                    BOOST_CHECK_EQUAL(num_xpubs, count_pks);
+                } else {
+                    BOOST_CHECK_EQUAL(script_provider_cached.origins.size(), count_pks);
                 }
             } else if (!(flags & MIXED_PUBKEYS)) {
                 // Only const pubkeys, nothing should be cached

Co-Authored-By: Andrew Chow <achow101-github@achow101.com>
@darosior
Copy link
Member Author

darosior commented Dec 9, 2021

Well, whatever. Amended the commit with your patch.

@darosior
Copy link
Member Author

Anything else that needs to be done there?

@achow101
Copy link
Member

ACK 36012ef

@achow101 achow101 merged commit e3ce019 into bitcoin:master Jan 20, 2022
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Jan 21, 2022
…pubkeys

36012ef qa: test descriptors with mixed xpubs and const pubkeys (Antoine Poinsot)

Pull request description:

  Writing unit tests for Miniscript descriptors i noticed that `test/descriptor_tests`'s `DoCheck()` assumes that a descriptor would either contain only extended keys or only const pubkeys: if it detects an xpub in the descriptor it would assert the number of cached keys is equal to the number of keys in the descriptor, which does not hold if the descriptor also contains const (raw?) public keys since we only cache parent xpubs.

ACKs for top commit:
  achow101:
    ACK 36012ef

Tree-SHA512: 2ede67a6dff726bcad3e260f3deb25c9b77542ed1880eb4ad136730b741014ce950396c69c7027225de1ef27108d609bafd055188b88538ace0beb13c7e34b0b
@bitcoin bitcoin locked and limited conversation to collaborators Jan 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants