Skip to content

fix: re-implement Psbt (de)serialization from/to readers/writers#3255

Merged
apoelstra merged 1 commit intorust-bitcoin:masterfrom
elsirion:2024-08-impl-reader-deserialization
Aug 28, 2024
Merged

fix: re-implement Psbt (de)serialization from/to readers/writers#3255
apoelstra merged 1 commit intorust-bitcoin:masterfrom
elsirion:2024-08-impl-reader-deserialization

Conversation

@elsirion
Copy link
Copy Markdown
Contributor

@elsirion elsirion commented Aug 27, 2024

Fixes #3250.

The serialization is less than ideal and still allocates a lot. I can understand not wanting to (ab)use the consensus encoding traits, but they have a pretty good interface, copying it and creating some EncodePsbt and DecodePsbt traits with similar interfaces would have been nice imo.

@elsirion elsirion marked this pull request as draft August 27, 2024 09:55
@github-actions github-actions bot added the C-bitcoin PRs modifying the bitcoin crate label Aug 27, 2024
@coveralls
Copy link
Copy Markdown

Pull Request Test Coverage Report for Build 10575860768

Details

  • 26 of 26 (100.0%) changed or added relevant lines in 1 file are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage increased (+0.004%) to 82.762%

Files with Coverage Reduction New Missed Lines %
bitcoin/src/psbt/serialize.rs 1 96.3%
Totals Coverage Status
Change from base Build 10567848333: 0.004%
Covered Lines: 19651
Relevant Lines: 23744

💛 - Coveralls


const PSBT_SERPARATOR: u8 = 0xff_u8;
if bytes.get(MAGIC_BYTES.len()) != Some(&PSBT_SERPARATOR) {
let separator: u8 = Decodable::consensus_decode(r)?;
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be slightly better to just read 5 bytes at once and compare them in one go.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can do that, although I think this version is slightly more readable.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that error[E0277]: the trait bound [u8; 5]: Decodable is not satisfied, I'd have to use read_exact like so:

diff --git a/bitcoin/src/psbt/serialize.rs b/bitcoin/src/psbt/serialize.rs
index 9fa465d1..d201ff45 100644
--- a/bitcoin/src/psbt/serialize.rs
+++ b/bitcoin/src/psbt/serialize.rs
@@ -81,15 +81,16 @@ pub fn deserialize(mut bytes: &[u8]) -> Result<Self, Error> {
     /// Deserialize a value from raw binary data read from a `BufRead` object.
     pub fn deserialize_from_reader<R: io::BufRead>(r: &mut R) -> Result<Self, Error> {
         const MAGIC_BYTES: &[u8] = b"psbt";
+        const PSBT_SERPARATOR: u8 = 0xff_u8;
+
+        let mut magic_and_separator: [u8; 5] = [0; 5];
+        r.read_exact(&mut magic_and_separator)?;
 
-        let magic: [u8; 4] = Decodable::consensus_decode(r)?;
-        if magic != MAGIC_BYTES {
+        if &magic_and_separator[..4] != MAGIC_BYTES {
             return Err(Error::InvalidMagic);
         }
 
-        const PSBT_SERPARATOR: u8 = 0xff_u8;
-        let separator: u8 = Decodable::consensus_decode(r)?;
-        if separator != PSBT_SERPARATOR {
+        if magic_and_separator[4] != PSBT_SERPARATOR {
             return Err(Error::InvalidSeparator);
         }
 

I don't think that's an improvement overall (especially since you already force buffered readers, which should make the benefit of not doing a one-byte read negligible).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, that makes sense. I have a plan to compromise between buffered and unbuffered reading but that's outside of scope of this PR.

@elsirion elsirion marked this pull request as ready for review August 27, 2024 10:05
Copy link
Copy Markdown
Collaborator

@Kixunil Kixunil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK cf129ad

@Kixunil
Copy link
Copy Markdown
Collaborator

Kixunil commented Aug 27, 2024

I think with push_decode we will just return an opaque encoder/decoder and have a helper methods to do the encoding/decoding directly so there won't be allocation or that much duplication.

Copy link
Copy Markdown
Member

@apoelstra apoelstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK cf129ad successfully ran local tests; LGTM -- I believe this is non-breaking, as does cargo-semver-checks, so we can backport this to 0.32

@apoelstra apoelstra merged commit 98252f3 into rust-bitcoin:master Aug 28, 2024
@tcharding tcharding added the port-0.32.x Needs porting to 0.32.x branch label Aug 28, 2024
@tcharding
Copy link
Copy Markdown
Member

Backported in #3266

@elsirion elsirion deleted the 2024-08-impl-reader-deserialization branch August 29, 2024 09:04
apoelstra added a commit that referenced this pull request Sep 2, 2024
…/writers

d5c79fd fix: re-implement (de)serialization from/to readers/writers (elsirion)

Pull request description:

  Backportt #3255 to `bitcoin v0.32`.

  One change is different from the original patch, import `DisplayHex` and remove imports of `String` and `Vec`.

ACKs for top commit:
  apoelstra:
    ACK d5c79fd successfully ran local tests

Tree-SHA512: 36bb4e8d12bde3f310291b4f066ef0ab17d121fcf1afe4a10ca6f8c29b3bd084d616c212a761e05d6d36255bb84059d4536109a4300208fca66e0752071649a1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C-bitcoin PRs modifying the bitcoin crate port-0.32.x Needs porting to 0.32.x branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Psbt no longer supports decoding from reader

5 participants