Skip to content

Add a test parsing transaction with a huge witness#1359

Merged
sanket1729 merged 1 commit intorust-bitcoin:masterfrom
Kixunil:parse-huge-witness
Nov 4, 2022
Merged

Add a test parsing transaction with a huge witness#1359
sanket1729 merged 1 commit intorust-bitcoin:masterfrom
Kixunil:parse-huge-witness

Conversation

@Kixunil
Copy link
Copy Markdown
Collaborator

@Kixunil Kixunil commented Nov 1, 2022

This transaction broke past versions of rust-bitcoin and LND so this adds a test to avoid reintroducing the problem in the future.

See also romanz/electrs#783

I'm publishing this immediately for research purposes. I can clean it up later if required (low on time rn) or we may even not merge it if there is a better test.

This transaction broke past versions of `rust-bitcoin` and LND so this
adds a test to avoid reintroducing the problem in the future.

See also romanz/electrs#783
@romanz
Copy link
Copy Markdown
Contributor

romanz commented Nov 1, 2022

IIUC, the failure is due to byte_size being 12000072 = 500003 (len) * 24 (mem::size_of::<$type>):

                let byte_size = (len as usize)
                                    .checked_mul(mem::size_of::<$type>())
                                    .ok_or(self::Error::ParseFailed("Invalid length"))?;
                if byte_size > MAX_VEC_SIZE {
                    return Err(self::Error::OversizedVectorAllocation { requested: byte_size, max: MAX_VEC_SIZE })
                }

and indeed the witness definition has changed between 0.27.1 and 0.28.0:

diff --git a/src/blockdata/transaction.rs b/src/blockdata/transaction.rs
index 4ecac18..445da7d 100644
--- a/src/blockdata/transaction.rs
+++ b/src/blockdata/transaction.rs
@@ -196,7 +198,7 @@ pub struct TxIn {
     /// Encodable/Decodable, as it is (de)serialized at the end of the full
     /// Transaction. It *is* (de)serialized with the rest of the TxIn in other
     /// (de)serialization routines.
-    pub witness: Vec<Vec<u8>>
+    pub witness: Witness
 }

The fix was introduced in 2fd0125.

@TheBlueMatt
Copy link
Copy Markdown
Member

Damn, ideally this kind of thing would have been addressed through #1023.

@TheBlueMatt
Copy link
Copy Markdown
Member

Ah, this is about rust-bitcoin 0.28, the above landed in 0.29 (according to github). So hopefully we have a substantially more robust answer to these kinds of bugs in 0.29 already.

@Kixunil
Copy link
Copy Markdown
Collaborator Author

Kixunil commented Nov 1, 2022

AFAIU even 0.28 is unaffected. This PR just adds test out of caution.

Copy link
Copy Markdown
Member

@elichai elichai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK, this is an important test, as this tx did break production systems

@sanket1729
Copy link
Copy Markdown
Member

I could not foresee the impact when I opened #997. It also lists other potential ways in which block decoding could fail, but all of them required the miner to specifically craft a non-standard tx. Perhaps, this is a lesson for us for back-porting all bug fixes related to decoding/encoding even if they "require miner assistance to deploy".

@sanket1729
Copy link
Copy Markdown
Member

sanket1729 commented Nov 1, 2022

Upon more investigation, this issue is not completely fixed in 0.28. The PR that changes Witness only accidentally fixes it for 0.28 while decoding things with a large Witness. #997 lists other things that are still possible to crash with 0.28. This is cleanly fixed in 0.29 with #1023.

@sanket1729
Copy link
Copy Markdown
Member

I have created #1360 as a backport for 0.28 users.

Copy link
Copy Markdown
Member

@tcharding tcharding left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK d6ca7e4

TheBlueMatt added a commit that referenced this pull request Nov 2, 2022
a0489d4 fuzz: use travis-fuzz.sh in CI (Andrew Poelstra)
4c6f9b3 fuzz: remove mysteriously-not-necessary quotes from gh action script (Andrew Poelstra)
7baa21c fuzz: disable features in honggfuzz (Andrew Poelstra)
e003e57 Add `consensus_decode_from_finite_reader` optimization (Dawid Ciężarkiewicz)

Pull request description:

  Backport for #1023. Required for #997. Addresses issues like #1359

ACKs for top commit:
  tcharding:
    ACK a0489d4
  TheBlueMatt:
    ACK a0489d4.

Tree-SHA512: 9145d9666e35ae77598aaecf89222c7234637b57ded39b69fbb93ee9ce01c6d7c938b36a2d86319ba84155f2424e524386593d6c0d7af449be6bd118f729fd64
@sanket1729 sanket1729 merged commit 2be133c into rust-bitcoin:master Nov 4, 2022
@Kixunil Kixunil deleted the parse-huge-witness branch November 4, 2022 09:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants