Make PsbtSigHashType use the same formatting as other *SigHashTypes#898
Make PsbtSigHashType use the same formatting as other *SigHashTypes#898apoelstra merged 3 commits intorust-bitcoin:masterfrom BP-WG:psbt/display-sighashtype
Conversation
There was a problem hiding this comment.
This PR highlights that we have some non-uniformity with the serde for the three sighash types
EcdsaSigHashTypeuses strings (e.g.SIGHASH_ALL)SchnorrSigHashTypeuses the variant name, this is almost certainly a bug (example below)PsbtSigHashTypeuses strings forEcdsaSigHashTypeinner type and hex strings forSchnorrSigHashTypeand non-standard sighash value.
Example serializatiion of SchnorrSigHashType:
let ty = SchnorrSigHashType::SinglePlusAnyoneCanPay;
let ser =serde_json::to_string(&ty).unwrap();
println!("{}", ser);
Outputs: "SinglePlusAnyoneCanPay"
Perhaps this PR should fix all that? (I swear I did this already but I cannot find the patches in my tree.)
EDIT: I found it, changes to SchnorrSigHashType are in the open RC-fix PR: 609a04a
src/util/psbt/map/input.rs
Outdated
There was a problem hiding this comment.
I get what you are doing with the REVERT-ME in commit brief message but I get the feeling that, since this is the only use of this, that the code will just get fixed and the commit will be left in there. I also just don't like the idea of having a commit with REVERT-ME in the brief message, it is going to scream at any dev that sees it making them check whats going on.
There was a problem hiding this comment.
So what is the suggested way of doing this?
|
@tcharding force-pushed version which uses your proposal for |
sanket1729
left a comment
There was a problem hiding this comment.
Did a superficial review, left one comment. Will do a detailed review after that is addressed. It might be worth looking at how bitcoin core psbt RPCs deal with sighash types
src/util/psbt/map/input.rs
Outdated
There was a problem hiding this comment.
What happens in the case when we have SchnorrSigHashType::Default
There was a problem hiding this comment.
This is not related to this PR and a question to how ecdsa_hash_ty function is working. Will check later, but I assume the correct behaviour should be to return ALL value.
There was a problem hiding this comment.
I think this is related to this PR. The display for PsbtSigHashType should cover the 00 value case which is Default. Note that this variant is not present in EcdsaSigHashType
There was a problem hiding this comment.
I believe it gets output as 0x00 - I wondered the same thing :)
tcharding
left a comment
There was a problem hiding this comment.
LGTM (by the way, I am explicitly not acking in a way that triggers github approve while I'm still new to the project. All my reviews should be considered 'soft' reviews until I'm more experienced).
src/blockdata/transaction.rs
Outdated
There was a problem hiding this comment.
I was copying your work over in #903 and I used unrecognized instead of misspelled_ty because at another place in the code this error could be caused by a malformed hex string, 'misspelled_ty' doesn't seem to fit that use case.
35b682d Implement Display/FromStr for SchnorrSigHashType (Tobin Harding) 46c4164 Improve SigHashTypeParseError field (Tobin Harding) c009210 Use full path for String in macro (Tobin Harding) Pull request description: Implement Display/FromStr for SchnorrSigHashType We currently implement `Display` and `FromStr` on `EcdsaSigHashType` and use them in the `serde_string_impl` macro to implement ser/de. Mirror this logic in `SchnorrSigHashType`. Patch 1 and 2 are preparatory patches for patch 3. ## Notes to reviewers This PR has some conflicts with #898 but is pushing in the same direction, I'm happy to let 898 go in first and rebase on top. ACKs for top commit: sanket1729: ACK 35b682d. Thanks, much easier to review now that the diff is small dr-orlovsky: ACK 35b682d Tree-SHA512: 481f192a3064ff39acf8904737dfb25b54ef128a37e0ca765ebb39138edac772d4f01ed10aa98ff185a8ed5668d64fa5d5957206b920ffe87950cafcf5a3b516
|
Re-based on top of merged #903 and optimized the code to support schnorr sighash types |
|
Now to/from string doesn't roundtrip. I think we want the following test to pass, right? The original version you had but with the schnorr optimizations works. |
|
@tcharding thank you for catching this... Overall PSBT lacks unit tests and it was my bad that I didn't wrote one for added functionality :( Updated with more tests and the problem fixed. |
|
Failing test shows ambiguity in handling Schnorr-specific sighash types in PSBT sighash type. So, should we support parsing |
|
@dr-orlovsky, is this really needed for release? We can this formatting next release. |
|
@sanket1729 RC1 introduced buggy code which will require major release to fix, i.e. at least half of year :( |
|
I force-pushed the way I propose to handle the ambiguity described in #898 (comment) |
|
@dr-orlovsky What is the bug here? Why can't PsbtSigHashType have different formatting? I agree that uniform encoding is good to have, but is it a bug if this is different formatting? |
src/util/psbt/map/input.rs
Outdated
There was a problem hiding this comment.
| fn psbt_sighash_type_schnorr_nostd() { | |
| fn psbt_sighash_type_schnorr_nonstd() { |
Non-std, right?
I appreciate you trying to expedite the release @sanket1729, if I understand correctly, @dr-orlovsky is working under the assumption that our release process is a little messed up at the moment and is worried that the 0.29 release won't be for another Perhaps we should remember 'perfect is the enemy of good' in relation to our processes right now, lets keep moving forwards but be forgiving of the current situation. Is that a fair interpretation of both of your positions? |
|
@tcharding , it is a fair characterization of m situation. |
|
AFAIU this PR does not work for the most common case of |
src/util/psbt/map/input.rs
Outdated
There was a problem hiding this comment.
I would like to see this work for Default too.
There was a problem hiding this comment.
This is the topic I raised above: what to do with SchnorrSigHash-specific types which must not be used with pre-taproot inputs (#898 (comment))
|
@sanket1729 my point is that wallets using JSON/YAML encoding of PSBTs (for instance I have one) will break backward compatibility when we will change this encoding in the next release. So this has to be a major release, i.e. it will take a long time. It is a bug since previously we had a different serde encoding, so we are breaking the compatibility - to break it again in the next version (introduction of a breaking incorrect behavior subjected to the future breaking changes = bug). |
This is unrelated to the issue, but I think this might be difficult for rust-bitcoin to guarantee. Fields name changes and lots of auto-derives break things constantly. I think you are misusing psbts if you are reliant on some other serde format for compatibility. Psbt is already compatible with all others that don't even understand serde/rust using consensus encoding (consensus::encode and consensus::decode). You should be using the most common psbt bas64 encoding. You can be sure that all major/minor versions of rust-bitcoin will stay true to consensus encoding.
I think we should do those as strings for now.
The point of breaking release is that structure of the objects change and the serde implementation will change! Here it is possible to fix it in a (somewhat) backward-compatible way. But this is not always possible and we cannot offer this guarantee. |
src/util/psbt/map/input.rs
Outdated
There was a problem hiding this comment.
I think this requires it's own custom code as this is the unification of both sighash types. You can write FromStr accordingly. We should not serialize RESERVED (as there is no such thing). And error on things that are not strings
impl fmt::Display for PsbtSigHashType {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
let s = match self.as_u32() {
0 => "SIGHASH_DEFAULT"
1 => "SIGHASH_ALL",
.. => "SIGHASH_NONE",
.. => "SIGHASH_SINGLE",
..=> "SIGHASH_ALL|SIGHASH_ANYONECANPAY",
.. => "SIGHASH_NONE|SIGHASH_ANYONECANPAY",
.....
};
f.write_str(s)
}
}
I understand your point and is well aware of Base64 and binary PSBT encodins as the main way to maintain the compatibility. But there are much more usecases, which can't work with these encodings and will require serde:
Regarding the question of whether serde serialization change is a breaking change requiring major version update, in previous PRs affecting it we had this discussion and AFAIR @apoelstra and @Kixunil pointed out that fact (I was not aware of it before). Not sure I will be able to find that place though... |
|
@sanket1729 |
|
For psbt I believe I try to always do it as a consensus encode string
because usually I want the tooling outside of rust to be able to work with
it.
My preferred solution would be to always do psbts, even for JSON, as
consensus encode.
…On Mon, Mar 28, 2022, 2:48 AM Dr. Maxim Orlovsky ***@***.***> wrote:
@sanket1729 <https://github.com/sanket1729>
Actually I assume that this PR covers another bug dealing with the failed
FromStr/Display roundtrips in SchnorrSigHashType: 4cf2dcf
<4cf2dcf>
—
Reply to this email directly, view it on GitHub
<#898 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGYN62JYY7SXV2BVAWXWV3VCF57LANCNFSM5RKBH3NA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Yes, changing serde serialization is a breaking change and will require a major rev. But as others have said, the serde serialization of PSBT really doesn't matter. There is a consensus encoding and that's what people should be using if they expect consistency (and if they expect to interoperate with any non-rust-bitcoin project). Probably the serde encoding should just encode the PSBT as either a base64 string or a big byte blob.. |
I will be concept NACK this since w/o serde there will be no simple way of converting PSBT into JSON or YAML for inspection/UI tasks Personally I am fine with having breaking changes even each two weeks, I am updating my code constantly. My issue was more about not waiting for 1/2 year+ for simple things to get done because we have to wait for the major release. |
|
@sanket1729, @tcharding fixed according to your suggestions |
|
I think you can still have a to/from Json method if you need a human
readable thing.
Also for human readable things the bitcoin js stuff can read a psbt for you
and then you can access it programmatically.
You can also throw the psbt at core and do decodepsbt or analyzepsbt.
But in general, I think the as a Json serialization is useless for interop
with most things, and we should drop it in favor of one that integrates
with any environment that can parse a psbt.
…On Mon, Mar 28, 2022, 7:36 AM Dr. Maxim Orlovsky ***@***.***> wrote:
Probably the serde encoding should just encode the PSBT as either a base64
string or a big byte blob..
I will be concept NACK this since w/o serde there will be no simple way of
converting PSBT into JSON or YAML for inspection/UI tasks
Personally I am fine with having breaking changes even each two weeks, I
am updating my code constantly. My issue was more about not waiting for 1/2
year+ for simple things to get done because we have to wait for the major
release.
—
Reply to this email directly, view it on GitHub
<#898 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGYN637ZT47BPX44JFDX7TVCG7W5ANCNFSM5RKBH3NA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
If serde implementation parses to Base58 string, this would not help.
I personally work with YAML from command-line a lot (bitcoin core's JSON is terrible for humans), and for me having command-line tool presenting given PSBT from Base58 like this is very useful: an example of PSBT YAML |
sanket1729
left a comment
There was a problem hiding this comment.
ACK 992857a. This is much better
|
Might make sense to add a similar PR to rus-bitcoin. https://github.com/rust-bitcoin/rust-secp256k1/pull/369/files |
|
Sure, though I again point out that "serde encoding" is not defined by us and can't possibly be "the same as consensus encoding". |
|
still doesn't seem like good default behavior if it's what you want. i would recommend doing this then: And either manually implementing Serialize/Deserialize OR doing a TryFrom serde field with a JSON. That way if you ever want the humanPSBT impl, just wrap the type in your struct / wrap before serializing if just sending a PSBT itself. but i don't think we want this in most situations, to the point where I definitely have conversion structs in places where i have to stringify psbts before serdeing, and that should be the default. In general, we should aim to use external stable serializations where they exist, and PSBT is such a case. |
The newly introduced
PsbtSigHashTypeuses very different serde formatting from previously usedEcdsaSigHashType; for instance it does not output human-readable sighash. This is especially obvious when printing out PSBT as JSON/YAML object and is a breaking change from the0.27. Serde human-readable implementation requiresDisplay/FromStr, which were also absent.