Skip to content

Prepare to enforce MAX_MONEY invariant#4164

Merged
apoelstra merged 10 commits intorust-bitcoin:masterfrom
tcharding:03-03-prepare-max-money
Mar 12, 2025
Merged

Prepare to enforce MAX_MONEY invariant#4164
apoelstra merged 10 commits intorust-bitcoin:masterfrom
tcharding:03-03-prepare-max-money

Conversation

@tcharding
Copy link
Copy Markdown
Member

We want to start enforcing MAX_MONEY as an invariant in the amount types. There are a few more steps we can do first to make that change easier to review.

@github-actions github-actions bot added C-bitcoin PRs modifying the bitcoin crate C-units PRs modifying the units crate test labels Mar 2, 2025
@tcharding
Copy link
Copy Markdown
Member Author

After this goes in we can undraft #4157.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 2, 2025

🚨 API BREAKING CHANGE DETECTED

To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section.

@github-actions github-actions bot added the API break This PR requires a version bump for the next release label Mar 2, 2025
/// Caller to guarantee that `satoshi` is within valid range.
///
/// See [`Self::MIN`] and [`Self::MAX_MONEY`].
pub const fn from_sat_unchecked(satoshi: i64) -> SignedAmount { SignedAmount(satoshi) }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm starting to think from_sats_unchecked should just be removed once unwrap() can be used in const context.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we've agreed elsewhere on removing it. If anyone wants to keep it I want it to be unsafe.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or at least make the constructor private. I'm fine with having the checked function in encapsulated module.

/// Computes the ceiling so that the fee computation is conservative.
impl ops::Mul<FeeRate> for Weight {
type Output = Amount;
type Output = NumOpResult<Amount>;
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: this change makes the docs in result stale. After #4179 goes in include a change that updates the docs since NumOpResult is now used in ops impls that return an Amount not just in ops on amounts.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since I had to rebase to fix the review suggestions I just grabbed the new result rustdocs from #4179 (comment) and rolled it into the fee change in 89751fc

@tcharding tcharding force-pushed the 03-03-prepare-max-money branch from 0742f85 to 5f0d694 Compare March 3, 2025 23:09
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 3, 2025

🚨 API BREAKING CHANGE DETECTED

To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section.

@tcharding tcharding mentioned this pull request Mar 3, 2025

#[test]
fn from_int_btc() {
let sat = Amount::from_sat;
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, we can later just change this to |x| Amount::from_sat(x).unwrap.
But it's repeated in multiple tests so why not just go all the way and define the function only once for the whole module? We can then just change one definition and it'll work fine.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries, I can do that.

pub const MAX_MONEY: Self = SignedAmount::from_sat_unchecked(21_000_000 * 100_000_000);
/// The minimum value of an amount.
pub const MIN: Self = SignedAmount(-21_000_000 * 100_000_000);
pub const MIN: Self = SignedAmount::from_sat_unchecked(-21_000_000 * 100_000_000);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we've agreed to remove the unchecked constructor we shouldn't use it here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to leave the _unchecked discussion out of scope for this PR because I didn't follow the conversation closely. Also since we cannot use a fallible constructor in const AFAIK then it seems reasonable ATM to use the _unchecked constructor. Is this too much code churn?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The result of that discussion was to remove the function. You don't have to make the change right now but adding more calls to it if we know we will change them for sure just adds noise for no reason. We're better off just dropping the commit that adds them, because those lines will change in any case.

we cannot use a fallible constructor in const AFAIK

We can for a while, our MSRV allowed that like year ago if not more. But also we don't have any fallible constructor in this commit so this is non-issue.

Is this too much code churn?

Yes, because that churn has literally zero value. So churn-to-value ration approaches infinity.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But also we don't have any fallible constructor in this commit so this is non-issue.

Come on now bro, this whole PR is preparation for enforcing the MAX_MONEY invariant - you are having a lend of me.

Just to re-cap. The amount module is currently borked. Normal patching procedure does not apply because we are not trying to maintain the non-existent broken-ness, nor are we trying to maintain a releaseable state.

The new sanity policy is to have one public constructor that accesses the inner field. So if we are leaving the _unchecked issue out of scope for this PR it makes logical sense to use from_sat_unchecked as the one public function.

Furthermore I don't know what you mean about using a fallible constructor in const for associated consts, there is no unwrap in const and we can't touch the inner field directly?

It is extremely difficult to go from the current broken state to the state we want to release without doing many many changes. I've done my best to make the changes at least discreet and kind of sane. I expected there to be a shit fight to get this in but 'don't change that line if we are going to change it again later' is just a pain in the arse. And I've been breaking it up and re putting it together for three months. Can we just merge something and iterate please.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Furthermore I don't know what you mean about using a fallible constructor in const for associated consts, there is no unwrap in const and we can't touch the inner field directly?

Well, you can still workaround the missing unwrap, right? For example, consts can be written as:

    pub const ZERO: Self = match SignedAmount::from_sat(0) {                                                                                                               │   |
        Ok(s) => SignedAmount::ZERO,                                                                                                                                       │75 |     pub const ZERO: Self = match SignedAmount::from_sat(0) {
        Err(e) => panic!("oops")                                                                                                                         │   |     ^^^^^^^^^^^^^^^^^^^^
    };

Would be nice if we could just bump to 1.83 and avoid that since it's an ugly hack. I don't think we should add it though and then pull it later because that's kind of annoying to deprecate for downstream imo.

Copy link
Copy Markdown
Member Author

@tcharding tcharding Mar 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct but that is way too ugly for associated consts IMO for very little additional value.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct but that is way too ugly for associated consts IMO for very little additional value.

We already have a lot of this going on with the understanding that it's temporary until things are available in MSRV

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I object to uselessly modifying the same bunch of lines twice. It's not helpful in any way to change these to use the _unchecked version if we know for sure we will be changing them back. It just wastes review time.

And regarding unwrapping in consts, just define a private const fn unwrap<T>(opt: Option<T>) -> T and then type unwrap(Amount::from_sat(whatever)). Or maybe consider the macro with compile-time check I suggested.

/// ```
#[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct SignedAmount(i64);
mod encapsulate {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea.

/// Caller to guarantee that `satoshi` is within valid range.
///
/// See [`Self::MIN`] and [`Self::MAX_MONEY`].
pub const fn from_sat_unchecked(satoshi: i64) -> SignedAmount { SignedAmount(satoshi) }
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or at least make the constructor private. I'm fine with having the checked function in encapsulated module.

match parse_signed_to_satoshi(s, denom).map_err(|error| error.convert(true))? {
// (negative, amount)
(false, sat) if sat > SignedAmount::MAX.to_sat() as u64 => Err(ParseAmountError(
(false, sat) if sat > SignedAmount::MAX_MONEY.to_sat() as u64 => Err(ParseAmountError(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

::MAX looks a lot like u64::MAX and for Bitcoin devs an amount is intrinsically a 64 byte unsigned integer.

Maybe, but not for Rust devs, where MAX represents maximum value a type can hold. I find it an argument for MAX.

The compiler will prevent any mixups. But we should deprecate anything in older versions that unconditionally produces amounts > 21M BTC.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I accidentally commented on the followup PR, although I agree that MAX is better than MAX_MONEY here.

The compiler will prevent any mixups. But we should deprecate anything in older versions that unconditionally produces amounts > 21M BTC.

Are you suggesting deprecating parse_signed_to_satoshi? It would be a good idea to look into modifying the function so it doesn't produce values greater than 21 BTC. Although, it's private so no need to deprecate and also that would slow down this PR so better to look into it later prob.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That one doesn't produce high amounts unconditionally.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So 2 votes for using MAX and not MAX_MONEY. @apoelstra do you want to weigh in, if not I'll change it to that.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW in any case I think we'll need to deprecate Amount::MAX in the old version saying it's creating absurdly large amounts and its semantics will change.

Copy link
Copy Markdown
Contributor

@yancyribbens yancyribbens Mar 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only use MAX and at the same time deprecate MAX - you've lost me?

I think he's pointing out that if you're using Amount::MAX for values greater than MAX_MONEY now, and the the API changes to limit Amount::MAX to MAX_MONEY that could be a surprise (since this const is public).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah MAX is fine.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now uses MAX everywhere and references this thread in the commit log to justify the change.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yancyribbens exactly!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is the possibility for catastrophe changing MAX from u64::MAX to MAX_MONEY without deprecation. Someone could be using this constant thinking that what they are getting is u64::MAX doing some computations etc. After upgrading they are going to get a completely different result without any warning. However, that change to MAX was done a while ago: #3719

pub const fn from_int_btc_const(whole_bitcoin: i64) -> SignedAmount {
match whole_bitcoin.checked_mul(100_000_000) {
#[allow(clippy::missing_panics_doc)]
pub const fn from_int_btc_const(whole_bitcoin: i32) -> SignedAmount {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR, this bad name got in and IIRC I was complaining about it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not to pile on with critiques, although I don't know why whole_bitcoin. But that doesn't pertain to this PR.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We likely all need to go over the amount module again with a fine tooth comb and a bunch of issues will fall out. Can we leave rename till then?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, this was just a side note.

Some(amount) => SignedAmount::from_sat(amount),
None => panic!("checked_mul overflowed"),
None => panic!("cannot overflow in i64"),
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it never overflows, might as well use *. But it'll still panic later.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it never overflows,

It can over flow. 21 million * 100000000 is greater than i32::MAX. It would _not_overflow had this been left as i64.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

? It's literally converted at the beginning.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see. Sorry I was looking at the function as it exists on master before this change.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, good call that this won't overflow so the checked_mul isn't needed.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it never overflows, might as well use *. But it'll still panic later.

I swear last time you said "might as well use checked arithmetic to make it explicit that it can't overflow" - either way its super easy to later change in a smaller PR - can we leave it as is?

Copy link
Copy Markdown
Member Author

@tcharding tcharding Mar 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I myself have done backflips like that in subsequent reviews so no hard feelings!

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I swear last time you said "might as well use checked arithmetic to make it explicit that it can't overflow"

I think we're talking apples and oranges. I'm certain I definitely didn't say such thing about arithmetic operators. I think what you have in mind is my suggestion to use Amount::from_sat(42).unwrap() in tests rather than unwrap_unchecked but in this case we're not handling the value returned from from_sat but the value that goes into it.

But yes, we can keep it.

/// per bitcoin overflows an `i64` type.
pub const fn from_int_btc_const(whole_bitcoin: i64) -> SignedAmount {
match whole_bitcoin.checked_mul(100_000_000) {
#[allow(clippy::missing_panics_doc)]
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can probably avoid this by using *.

/// broadcastable on today’s Bitcoin network.
#[deprecated(since = "0.32.0", note = "use `minimal_non_dust` etc. instead")]
fn dust_value(&self) -> Amount { self.minimal_non_dust() }
fn dust_value(&self) -> Option<Amount> { self.minimal_non_dust() }
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't we agree that these are ridiculously unlikely and not worth it? I can't remember.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't we agree that these are ridiculously unlikely and not worth it

Since it's already here, I don't see a downside to being cautious here.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can only happen if something is attempting to produce a script that is 175000000 times longer than the block size. To my knowledge, everyone just uses one of the standard block templates, so it's valid by construction.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing it like this makes the next change (making from_sat fallible) way easier. Last time I did not do that and it turned out too messy. So this is way cleaner. We can always re-visit if we don't want the Option and prefer to panic.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does it make it way easier if you can simply use from_sat(x).unwrap()?

Anyway, don't take this as blocking, just curiosity.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its easier because one doesn't have to reason about what values cause a panic - which I did last time and still got wrong.

Anyway, don't take this as blocking, just curiosity.

Cool, thanks.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see. OK.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #4221

We should address this before release, but fine to leave it open during refactors.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah a downside to this change is it is disruptive.

@tcharding
Copy link
Copy Markdown
Member Author

The only change required after review IMO is using a function in the tests. Since this is a blocking PR that has been in the pipeline for nearly 3 moths can we just merge it like this and do a bunch of little follow ups please?

@Kixunil
Copy link
Copy Markdown
Collaborator

Kixunil commented Mar 4, 2025

The only change required after review IMO is using a function in the tests.

I really, really want to drop the unchecked commit and my point wrt single function was to make it single right away so that the later change will be just one-line and thus easier to review, so it makes more sense to do it now rather than later but it's also not a big deal.

I'm going to sleep anyway, so you have around 8 hours to drop the commit. I believe you an do it in that time. :)

@tcharding
Copy link
Copy Markdown
Member Author

Sure thing, I'll take a look. I'm going to Sydney to collaborate on PSBT stuff today but this is top of my TODO list.

@tcharding
Copy link
Copy Markdown
Member Author

I didn't get to this. Will do it tomorrow.

@tcharding
Copy link
Copy Markdown
Member Author

I added a pair of constructor functions to the test module instead of local aliases. It is much cleaner, thanks for pushing for that.

@tcharding tcharding force-pushed the 03-03-prepare-max-money branch from 5f0d694 to d3c2386 Compare March 5, 2025 22:01
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 5, 2025

🚨 API BREAKING CHANGE DETECTED

To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section.

@tcharding
Copy link
Copy Markdown
Member Author

tcharding commented Mar 5, 2025

MAX vs MAX_MONEY is left as is. I would like to make it out of scope for this PR. As mentioned in 8c31236, by using MAX_MONEY in this PR we set the stage for a simple one liner to change it to using MAX. This PR is blocking and MAX vs MAX_MONEY usage internally is a trivial thing.

Now uses MAX everywhere.

@tcharding
Copy link
Copy Markdown
Member Author

Come on crew, lets make some progress. units should have been done months ago. rust-bitcoin devs need to ship or else why are we here.

@tcharding tcharding mentioned this pull request Mar 5, 2025
tcharding added a commit to tcharding/rust-bitcoin that referenced this pull request Mar 6, 2025
Just use MAX everywhere in this codebase.

After discussion in PR consensus was to just use MAX throughout the
codebase.

ref: rust-bitcoin#4164 (comment)
@tcharding tcharding force-pushed the 03-03-prepare-max-money branch from d3c2386 to 3b9d0c1 Compare March 6, 2025 00:18
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 6, 2025

🚨 API BREAKING CHANGE DETECTED

To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section.

@tcharding
Copy link
Copy Markdown
Member Author

@apoelstra please consider merging this PR after 48 hours from the last push unless there is any review comments that indicate it is buggy. Everything else can be done as a follow up.

Copy link
Copy Markdown
Contributor

@jamillambert jamillambert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had a look through everything and didn't see any problems. I noted a couple of nits.

/// Constructs a new [`Amount`] with satoshi precision and the given number of satoshis.
///
/// Caller to guarantee that `satoshi` is within valid range. See [`Self::MAX`].
pub const fn from_sat_unchecked(satoshi: u64) -> Amount { Self(satoshi) }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub const fn from_sat_unchecked(satoshi: u64) -> Amount { Self(satoshi) }
pub const fn from_sat_unchecked(satoshi: u64) -> Amount { Amount(satoshi) }

There is a mixture of the use of Self:: and Amount:: or SignedAmount:: in signed, for no obvious reason.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self is generally better because it protects against renames of types or added generics and such. But I don't think it'll ever be done here. I don't think this code is worth changing.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was more of a general comment than pulling out that one occurrence. It's just one example where the same function in unsigned and signed are different pub const fn from_sat_unchecked(satoshi: i64) -> SignedAmount { SignedAmount(satoshi) }. The functions throughout seem to vary in which they use.

But it can be changed later, or not at all.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good, thanks. Promoted to #4210

assert_eq!(sp("-.5", btc), Ok(SignedAmount::from_sat_unchecked(-500_000_00)));
assert_eq!(p(&more_than_max, den_btc), Err(OutOfRangeError::too_big(false).into()));
assert_eq!(p("0.000000042", den_btc), Err(TooPreciseError { position: 10 }.into()));
assert_eq!(p("1.0000000", den_sat), Ok(Amount::from_sat_unchecked(1)));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
assert_eq!(p("1.0000000", den_sat), Ok(Amount::from_sat_unchecked(1)));
assert_eq!(p("1.0000000", den_sat), Ok(sat(1)));

There are a few cases of this that were missed.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks man: #4211

@jamillambert
Copy link
Copy Markdown
Contributor

Sorry if this is a stupid question but:

Why do we even have an unsigned amount?

If it was to allow for infallible conversion between a u64 and an Amount this is now moot. A SignedAmount can be used in all cases an Amount can. Wouldn't it make sense to just remove unsigned and only have one amount that can be negative?

@apoelstra
Copy link
Copy Markdown
Member

@tcharding yeah, if you can't give me a green checkmark then that's the best option I think. Can you tweak the final commit in some trivial way and force-push? I should be able to run local-CI on the tweaked commit pretty quickly (possibly it won't redo any tests at all, if the source tree doesn't change..).

@tcharding
Copy link
Copy Markdown
Member Author

I just did an empty ammend (changed timestamp of last patch). Now GitHub thinks there are merge conflicts.

tcharding and others added 4 commits March 11, 2025 05:32
We have a const for this, use it.

Internal change only.
Throughout the `amount::tests` module we use `sat` and `ssat` as aliases
to amount constructors but in on test we use them as `Denomination`
variables. To assist clarity and so we can introduce uniform usage of
the constructor aliases change the variable names to use the `den_`
prefix.

Internal change only, no logic changes.
There is an as yet unresolved discussion about the unchecked amount
constructor. In an effort to focus the amount of changes required later
and also to make the `tests` module uniform use the `sat` and `ssat`
constructor functions everywhere.

Internal change only, no logic changes.
We are about to start enforcing the MAX_MONEY invariant. Doing so will
change constructors to return an error type.

In preparation use the `_unchecked` constructor for all the consts.

Internal change only, no logic changes.
tcharding added a commit to tcharding/rust-bitcoin that referenced this pull request Mar 10, 2025
Just use MAX everywhere in this codebase.

After discussion in PR consensus was to just use MAX throughout the
codebase.

ref: rust-bitcoin#4164 (comment)
@tcharding tcharding force-pushed the 03-03-prepare-max-money branch from 4bdd890 to c5601cf Compare March 10, 2025 18:34
@github-actions
Copy link
Copy Markdown

🚨 API BREAKING CHANGE DETECTED

To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section.

@tcharding
Copy link
Copy Markdown
Member Author

Conflicted with the docs PR that merged. I rebase and fixed the conflict. Its 5:30 am here but git range-diff tells me I did it correctly.

The unchecked-should-be-unsafe conversation is out of scope for this
patch. We want to bite off small chunks so the constructors are left as
they currently are - we are just doing the encapsulation here. This is
in preparation for enforcing the MAX_MONEY invariant which is not
currently enforced.

As per the sanity rules policy outline in:

 rust-bitcoin#4090

For both amount types create a private `encapsulate` module that
consists of exactly the type and a single constructor and a single
getter.
Just use MAX everywhere in this codebase.

After discussion in PR consensus was to just use MAX throughout the
codebase.

ref: rust-bitcoin#4164 (comment)
I royally botched the recent effort to make const amount constructors
use a smaller type. I left in an  unnecessary panic and forgot to do
both of them.

Note these function return values will change again very shortly when we
start enforcing the MAX_MONEY invariant. However the 64 to 32 bit change
is unrelated to that and is easier to review if done separately.

Whole bitcoin can not in any sane environment be greater than 21,000,000
which fits in 32 bits so we can take a 32 bit integer in the whole
bitcoin constructors without loss of utility. Doing so removes the
potential panic.

This is a breaking API change. We elect not to deprecate because we want
to keep the same function names.
Calculating the minimum non-dust fee currently panics if either the
script is really big or the dust fee rate is really big.

Harden the API by returning an `Option` instead of panicing.
Now that we have the `NumOpResult<Amount>` type that is used to show a
math calculation returned a valid amount we can use it when multiplying
weight and fee rates thus removing panics.
When we enforce the MAX_MONEY invariant these functions would require
the function signature changing - might as well just delete them.
@tcharding tcharding force-pushed the 03-03-prepare-max-money branch from c5601cf to 5d851f1 Compare March 10, 2025 18:39
@tcharding
Copy link
Copy Markdown
Member Author

Oh no I didn't - try now.

@github-actions
Copy link
Copy Markdown

🚨 API BREAKING CHANGE DETECTED

To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section.

Copy link
Copy Markdown
Contributor

@jamillambert jamillambert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 5d851f1

Copy link
Copy Markdown
Member

@apoelstra apoelstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 26d79a4; successfully ran local tests

Copy link
Copy Markdown
Member

@apoelstra apoelstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 5d851f1; successfully ran local tests

@apoelstra apoelstra merged commit 3cef539 into rust-bitcoin:master Mar 12, 2025
24 checks passed
erickcestari pushed a commit to erickcestari/rust-bitcoin that referenced this pull request Mar 12, 2025
Just use MAX everywhere in this codebase.

After discussion in PR consensus was to just use MAX throughout the
codebase.

ref: rust-bitcoin#4164 (comment)
mcelrath pushed a commit to braidpool/rust-bitcoin that referenced this pull request Mar 26, 2025
* Fix key/script spend detection in `Witness`

The `taproot_control_block` did not properly detect whether it deals
with script spend or key spend. As a result, if key spend with annex was
used it'd return the first element (the signature) as if it was a
control block.

Further, the conditions identifying which kind of spend it was were
repeated multiple times but behaved subtly differently making only
`taproot_control_block` buggy but the other places confusing.

To resolve these issues this change adds a `P2TrSpend` enum that
represents a parsed witness and has a single method doing all the
parsing. The other methods can then be trivially implemented by matching
on that type. This way only one place needs to be verified and the
parsing code is more readable since it uses one big `match` to handle
all possibilities.

The downside of this is a potential perf impact if the parsing code
doesn't get inlined since the common parsing code has to shuffle around
data that the caller is not intersted in. I don't think this will be a
problem but if it will I suppose it will be solvable (e.g. by using
`#[inline(always)]`).

The enum also looks somewhat nice and perhaps downstream consumers could
make use of it. This change does not expose it yet but is written such
that after exposing it the API would be (mostly) idiomatic.

Closes rust-bitcoin#4097

* Add a test case checking `taproot_control_block`

The previous commit fixed a bug when `taproot_control_block` returned
`Some` on key-spends. This adds a test case for it which succeeds when
applied after the previous commit and fails if applied before it.

* Add `taproot_leaf_script` methood to `Witness`

We already have `tapscript` method on `Witness` which is broken because
it doesn't check that the leaf script is a tapscript, however that
behavior might have been intended by some consumers who want to inspect
the script independent of the version. To resolve the confusion, we're
going to add a new method that returns both the leaf script and, to
avoid forgetting version check, also the leaf version.

This doesn't touch the `tapscript` method yet to make backporting of
this commit easier. It's also worth noting that leaf script is often
used together with version. To make passing them around easier it'd be
helpful to use a separate type. Thus this also adds a public POD type
containing the script and the version. In anticipation of if being
usable in different APIs it's also generic over the script type.
Similarly to the `tapscript` method, this also only adds the type and
doesn't change other functions to use it yet. Only the newly added
`taproot_leaf_script` method uses it now.

This is a part of rust-bitcoin#4073

* Add Timestamp newtype

Bitcoin block headers have a timestamp. Currently we are using a
`u32`. while this functions correctly it gives the compiler no chance
to enforce type safety.

Add a `Timestamp` newtype that is a thin wrapper around a `u32`.
Document it and test the API surface in `api.rs`.

* Fix `is_invalid_use_of_sighash_single()` incompatibility with Bitcoin Core

* Add test for sighash_single_bug incompatility fix

* Grab missing changelog

We release `v0.32.4` and `v0.32.5` already but forgot to merge the
changelog entries back into master.

Grab the missing changelog entries from the `0.32.x` release branch.

* Update input_string.rs

* Update key.rs

* Update transaction.rs

* Update sighash.rs

* chore: remove redundant words in CONTRIBUTING.md

Signed-off-by: costcould <fliter@myyahoo.com>

* Derive Copy for NumOpResult

The `NumOpResult` type is way more ergonomic to use if it derives
`Copy`. This restricts the `NumOpResult` to being `Copy` as well.

This does restrict what we can include in the error type in the future.

Derive Copy for `NumOpResult` and `NumOpResult`.

* units: introduce impl_op_for_references and use it in three places

This macro can generally handle a lot of different cases where we
implement "the same trait but on references". We introduce it here and
use it in two places. We will use it in many more, but I wanted to make
the diff small on this commit, which introduces the actual macro code
and might take a bit of reading to understand.

You may want to use --color-moved-ws=allow-indentation-change to review
this, and the next commit.

The next set of changes will mechanically delete other macros that are
made redundant by this.

* units: allow multiple invocations in impl_op_for_references macro

This is not too complicated a change to support and it will reduce the
noise in the following commits a fair bit.

* units: rearrange a bit of code to prep the next commit

The next commit changes a lot of code, but almost entirely by moving and
indenting it. We try to do the moves here ahead of time, so it the diff
for the next commit will be just deletions and indentations.

* units: replace a gazillion more macro calls with new macro

Looks like a large diff but if you run

    git show --color-moved-ws=allow-indentation-change

you will see that it's 100% moves (though moves of code into the
reference macro). May be easier to just look at src/amount/result.rs
after this; it's pretty short now.

* units: extend op reference macro to handle generics and where clauses

This is a bit ugly and requires that we put our where-clauses in
parentheses because the macro_rules parser sucks, but it allows us to
move the blanket-impls on NumOpResult into the macro.

This commit moves one instance and updates the macro; the next commits
will change the rest.

* units: pull generic op impls on NumOpResult into macro

* Add a few impls to the result macro

Add a few missing impls to the `impl_op_for_references` macro.

Includes a minor whitespace change so that traits are grouped together.

* Macroise the NumOpResult tests

Macroise the tests improving coverage along the way.

Includes a TODO to remind us to do `Neg` more fully.

* Hide relative locktime error internals

As part of the 1.0 effort and forward maintainability hide the internals
of the two error types in the `relative` locktime module. Doing so
allows us to remove the `non_exhaustive` attribute. Add getters to get
at the error innards.

* primitives: Hide script error internals

As part of the 1.0 effort and forward maintainability hide the internals
of the two error types in the `script` module. Add getters to get at the
invalid size.

* Move taproot back to bitcoin crate

I don't know what I was thinking when I move the taproot hash types to
`primitives`. As correctly pointed out by Kix we agreed to only have
blockdata in `primitives`.

Move the taproot hash types back to `bitcoin::taproot` and remove the
extension traits.

* fuzz: add coverage for Display for Script

* Policy: Relax MIN_STANDARD_TX_NONWITNESS_SIZE to 65

Align with Bitcoin Core's policy by reducing the minimum non-witness
transaction size from 82 to 65 bytes. This change allows for more
minimal transaction cases (e.g., 1 input with 1 OP_RETURN output),
while still maintaining protection against CVE-2017-12842.

Matches bitcoin/bitcoin#26398

* hashes: Remove Clone trait bound from Tag

Implementors of the Tag trait had to use the #[derive(Clone)] attribute.
This change eliminates this need by removing the Clone trait
bound from the Tag trait.

* Add support for pay to anchor outputs

Add support for the newly created Pay2Anchor output-type.

See bitcoin/bitcoin#30352

* Typo fix in: README.md

Hi,

I suggest some typo fix' for this doc:

1- "since these are needed to display hashes anway." 
Should be "since these are needed to display hashes anyway." (spelling error).

2- "bench mark" and "bench marks" Should be "benchmark" and "benchmarks" (incorrect spacing; "benchmark" is a single word).

Thanks.

* units: Improve code comment on macros

We have two macros that are hidden because they are code de-duplication
tools. However the output they produce is, and has to be, stable so that
we can use them in `units` and `primitives` without inadvertently
breaking semver in `primitives`.

* chore: fix some typos in comments

Signed-off-by: looklose <shishuaiqun@yeah.net>

* Update mod.rs

* Update serialized_signature.rs

* Update message.rs

* Update instruction.rs

* Automated update to Github CI to rustc nightly-2025-02-28

* 2025-03-02 automated rustfmt nightly

* update secp256k1 to 0.30.0

* Add tests for amount op int

We aim to support three ops on amount types that use an integer for the
right hand size. Prove that implement them.

* Improve add/sub tests for amount types

Add a few macros to test `Add` and `Sub` impls for both amount types,
all combos of type and res (eg `Amount` and `NumOpResult<Amount>`), and
all combos of references.

* Add more add/sub tests

Add units tests for various values including negative values.

* Test amount ops that involve an integer

From the amount types `Div`, `Mul`, and `Rem` can have an integer on one
side. Test them - includes commented out test for one missing combo.

* Add missing Mul impls for amount types

Add and test missing `Mul` impls for both amount types.

* Implement Div by amount for amount types

It is semantically valid to divide an amount by another amount. The
result of the operation is an integer.

Note that we cannot implement `Div` by `NumOpResult` because there is no
way to show the div by invalid case.

Implement `Div` by amount for both amount types.

* Add additional re-exports

As we do for other types add two new alias' at the crate root of
`primitives` and mirror it in `bitcoin`:

- `BlockVersion`
- `TransactionVersion`

* Move opcodes back to bitcoin

Duplicate `opcodes` in `bitcoin` and hide it in `primitives` so we do
not have to commit to the API.

We use opcodes in `impl fmt::Display for Script`.

Close: rust-bitcoin#4144

* units: Remove unnecessary code comments

These comments to not add much value - remove them.

* Move Assign impls together

Next patch will move all the impls of `Add` and `Sub` into a macro call.
In order to make that patch smaller move the assign impls to be together
below the add/sub impls.

Code move only, no logic change.

* Use impl_op_for_references for block height/interval

We have a new macro for implementing ops with a bunch of reference
combos. Lets use it for block `Height` and `Interval`.

This patch is strictly additive.

* Use impl_add/sub_assign for block interval

We have a macro for implementing `AddAssign` and `SubAssign`.

Use the macro, gets us an additional ref impl for both.

* Improve docs crate headings

I took a look at the rendered HMTL of `bitcoin`, `primitives`, `units`,
`serde`, and `tokio` and picked a header style that I thought looked
good.

Use it for `primitives` and `units`.

* hashes: Improve crate docs heading

Use same for as the other crates and emphasise that this is a Bitcoin
hashes crate.

* io: Improve crate docs heading

Use the same format as the other crates.

* chacha20: Add a docs heading

Like we do for the other crates add a heading to the crate docs.

* chacha20: Add whitespace

Mirror the other crates. Whitespace only.

* Improve Weight rustdocs

Do trivial improvement to rustdocs for the `Weight` type.

* Make Weigth::from_kwu const

The other two `Weight` constructors are const already.

* Use uniform return statement in docs

We have a bunch of 'Returns [`None`] if .. ' statements. Make the whole
module uniform.

* Manually update nightly version

While trying to use the `macro_use_imports` lint I found that there is a
bug in last weeks nightly. It has been fixed already so lets update.

Update to todays nightly compiler. Doing so causes some new linter
warnings, for now we just allow them.

* Inline small functions

Functions that just delegate or do trivial operations should be inline.

Add `#[inline]` to all functions in `primitives` that fit the criteria.

* fuzz: cover count_sigops{_legacy} for Script

* fuzz: cover minimal_non_dust for Script

* fuzz: move consume_random_bytes to a util file

* Fix the release script that checks for TBD

`release.sh` is missing `units`.

Add `units` to the list of crates to check.
Reorder the crates alphabetically.

* Implement Rem for Weight

`Weight` implements `Div` but not `Rem`

Add `Rem` impl for `Weight`

* Add a test for remainder

A remainder operation has been implemented for `Weight`.

Test the functionality of remainder with both a `Weight` and `u64`

* bitcoin: Remove hash type re-exports

The `{W}PubkeyHash` and `{W}ScriptHash` types are not likely to be used
directly by consumers of the library because we have other function that
return them and are more ergonomic to use. There is therefor no good
reason to re-export them from the crate root.

* Make `hex` in `internals` optional

The `hex` crate is not always desirable - e.g. when the consumer wants
to work with raw data only. We already had this optional in `hashes` but
if `hashes` is going to depend on `internals` it would break this
property.

This change makes `hash` optional, since it's easy: there's just one
struct that depends on it.

* Bump GitHub Actions Artifacts to v4

v3 is deprecated and causes an automatic fail of fuzz workflow.

Bump the version of `actions/upload-artifact` and
`actions/download-artifact` to v4.

* fuzz: add consume_u64

* fuzz: cover minimal_non_dust_custom  for Script

* ci: update Kani GitHub Actions runners to ubuntu-24.04

GitHub is deprecating the ubuntu-20.04 runner, with removal scheduled for
2025-04-01 (see: actions/runner-images#11101)
This commit updates workflow files to use ubuntu-24.04 instead.

* Don't panic in `PrivateKey::from_slice`

During upgrade of `secp256k1` a number of function calls needed to be
rewritten to not use `from_slice` but `from_byte_array`. Unfortunately,
the conversions wasn't correct and caused panics on invalid inputs
rather than errors.

This fixes it simply by calling `try_into` on the entire slice and
converting the error.

* Add a test checking `PrivateKey::from_slice`

This test checks the previous fix - if ordered before the previous
commit it will fail.

* Add `from_byte_array` to `PrivateKey`.

Private keys have statically-known length of 32 bytes and we are
migrating types with known lenths to use `from_byte_array` methods. This
adds the method to `PrivateKey` as well and uses it to implement
`from_slice`.

* Deprecate `PrivateKey::from_slice` method

Since arrays better convey the intention than slices when parsing
fixed-sized bytes we're migrating to them. This deprecates the
`from_slice` method similarly to how we do it elsewhere.

* Kill mutants in primitives and units

This kills 15 mutants found with the mutants workflow

* Fix bug in PSBT `Deserialize` for `XOnlyPublicKey`

During upgrade of `secp256k1` a number of function calls needed to be
rewritten to not use `from_slice` but `from_byte_array`. Unfortunately,
the conversion wasn't correct and caused panics on invalid inputs
rather than errors.

This fixes it simply by calling `try_into` on the entire slice and
converting the error.

* Add test checking `XOnlyPublicKey::deserialize`

This test checks the previous fix - if ordered before the previous
commit it will fail.

* Minor: fix typo

* ci: Update GitHub workflows to use ubuntu-24.04 instead of ubuntu-latest

* primitives: Enable pedantic lints

Enable all the pedantic lints and fix warnings.

Notable items:

- `enum_glob_used` import types with a single character alias
- `doc_markdown`: add a whitelist that includes SegWit and OpenSSL

* units: Prevent casting pub enums as ints

A public enum with simple variants gets an automatic integer variant
that can be cast by library consumers. This puts a unnecessary
maintenance burden upon us because we cannot then add variants in the
middle of others.

Add a hidden variant to the single public non-error enum in `units`.

* Add hash_again regression test

Add a simple regression test prior to patching the
`sha256::Hash::hash_again` function.

* Add Hash type and finalize method to HashEngine

Add an associated const `Hash` to the `HashEngine` trait. Also add a
`finalize` method that converts the engine to the associated hash.

For now just use the existent `from_engine` stuff. We can refactor
later.

* Bound HmacEngine on HashEngine

We would like to do away with the `GeneralHash` trait. Currently we
bound `Hmac` and `HmacEngine` on it but this is unnecessary now that we
have added `HashEngine::finalize` and `HashEngine::Hash`.

Bound the `HmacEngine` on `HashEngine` (which has an associated `Hash`
type returned by `finilalize`).

Bound `Hmac` type on `T::Hash` where `T` is `HashEngine`.

Includes some minor shortening of local variable names around hmac
engine usage.

Note this means that `Hmac` no longer implements `GeneralHash`.

* hashes: Add hash function to modules

Add a standalone `hash` function that is a drop in replacement for
`GeneralHash::hash`. Do not add it to `hmac` - this is in parity with
the current code because `Hmac` does not implement `GeneralHash::hash`.

Use the new function in `bitcoin` removing all occurrences of
`GeneralHash` from `bitcoin`.

In `hashes` replace usage of `GeneralHash::hash` with the new `hash`
function.

* hashes: Add hash_byte_chunks function to modules

 Add a standalone `hash_byte_chunks` function that is a drop in
 replacement for `GeneralHash::hash_byte_chunks`. Do not add it to
 `hmac` - this is in parity with the current code because `Hmac` does
 not implement `GeneralHash::hash_byte_chunks`.

* io: Use function in place of GeneralHashExt

We would like to remove the `GeneralHash` trait but we want to keep the
`hash_reader` functionality.

Add a stand alone function (when `hashes` is enabled) `hash_reader`.
Remove the extension trait.

* Remove unused trait import

This import is not needed. Interesting that the linter does not catch
it.

* hashes: Remove the GeneralHash trait

Now that we are able to unambiguously go from a hash engine to its
associated hash type there is no longer any need for the `GeneralHash`
trait.

Please note that IMO this concept of a general hash type as opposed to
one where one can hash arbitrary data still exists in the codebase - it
is implicitly in the `hash_newtype` macro.

Remove the `GeneralHash` trait.

* Take spent closure by value in count_witness_sigops and count_p2sh_sigops

This fixes issue rust-bitcoin#4141
Change count_witness_sigops and count_p2sh_sigops to take the spent
closure by value instead of &mut

- Changed both functions to accept S as a value (FnMut) instead of &mut S
- Removes need to annotate &mut when calling the function

* Replace uses of `chunks_exact` with `as_chunks`

In the past we've been using `chunks_exact` because const generics were
unstable but then, when they were stabilized we didn't use `as_chunks`
(or `array_chunks`) since they were unstable. But the instability was
only because Rust devs don't know how to handle `0` being passed in. The
function is perfectly implementable on stable. (With a tiny,
easy-to-understand `unsafe` block.) `core` doesn't want to make a
decision for all other crates yet but we can make it for our own crates
because we know that we simply never pass zero. (And even if we did, we
could just change the decision.)

It also turns out there's a hack to simulate `const {}` block in our
MSRV, so we can make compilation fail early.

This commit adds an extension trait to internals to provide the methods,
so we no longer have to use `chunks_exact`. It also cleans up the code
quite nicely.

* Add fee_rate::serde re-export

When we added the `fee_rate::serde` module we forgot to re-export it.
This is needed so downstream can do specify serde attributes on struct
fields.

```rust
    #[serde(with = "bitcoin::fee_rate::serde::as_sat_per_kwu")]
    rate: FeeRate,
```

* Fix some comments

Signed-off-by: NinaLua <iturf@sina.cn>

* Move module out of fuzz_target directory

The new module `fuzz_utils` in the `fuzz_targets/` directory causes
`verify-execution` to fail.

Move the module to the `src/` directory. Create a `lib.rs` file.

* Re-name Timestamp to BlockTime

We just added a `Timestamp` type without knowing that there was a push
by OpenTimestamps to also create a timestamp and that our new type may
lead to confusion. Our timestamp is explicitly for the `time` field in a
block so we can call it `BlockTime`. This name change makes the module
name stale but we will change that in a following patch to ease review.

* Rename timestamp module to time

We just re-named `Timestamp` to `BlockTime`. We have a `units::block`
module but it currently holds abstractions (`BlockHeight` and
`BlockInterval`) that are not onchain abstractions and therefore
somewhat different from the `BlockTime`. Instead of making `block` a
block 'utils' module instead re-name the `timestamp` module to `time`.

* Automated update to Github CI to rustc nightly-2025-03-06

* Remove warning section

Since monadic handling has been introduced, panics have been replaced
with return errors.  Therefore this section is no longer applicable.

* Replace underflow with overflow in doc comments

The use of underflow is misleading.  Adding one to MAX and
subtracting one from MIN are both considered an overflow.

* primitives: Feature gate import

Feature gate the `Infallible` import. Found with `clippy`.

* fix typos

* docs: The quotation marks are incorrect.

Signed-off-by: RiceChuan <lc582041246@gmail.com>

* docs: Update README to replace use of mutagen with cargo-mutants

* Remove references to cfg(mutate) from lint allow - no longer allowed

Also fix incorrect spelling of honggfuzz

* Add validation for private key format and master key constraints

This commit adds additional validation checks when decoding extended private keys:

1. Verifies that byte 45 is zero as required by BIP-32 specification
2. For master keys (depth=0), ensures parent fingerprint is zero
3. For master keys (depth=0), ensures child number is zero

These checks improve security by rejecting malformed keys that could
potentially lead to unexpected behavior. Added corresponding error types
and unit tests to verify each validation rule.

* psbt: Use Amount::ZERO in unit test

We have a const for this, use it.

Internal change only.

* Use den_ prefix for local Denomination variable

Throughout the `amount::tests` module we use `sat` and `ssat` as aliases
to amount constructors but in on test we use them as `Denomination`
variables. To assist clarity and so we can introduce uniform usage of
the constructor aliases change the variable names to use the `den_`
prefix.

Internal change only, no logic changes.

* Use sat/ssat constructors throughout tests

There is an as yet unresolved discussion about the unchecked amount
constructor. In an effort to focus the amount of changes required later
and also to make the `tests` module uniform use the `sat` and `ssat`
constructor functions everywhere.

Internal change only, no logic changes.

* Use _unchecked in amount const types

We are about to start enforcing the MAX_MONEY invariant. Doing so will
change constructors to return an error type.

In preparation use the `_unchecked` constructor for all the consts.

Internal change only, no logic changes.

* Enforce newtype sanity rules for amount types

The unchecked-should-be-unsafe conversation is out of scope for this
patch. We want to bite off small chunks so the constructors are left as
they currently are - we are just doing the encapsulation here. This is
in preparation for enforcing the MAX_MONEY invariant which is not
currently enforced.

As per the sanity rules policy outline in:

 rust-bitcoin#4090

For both amount types create a private `encapsulate` module that
consists of exactly the type and a single constructor and a single
getter.

* Pick one - MAX or MAX_MONEY

Just use MAX everywhere in this codebase.

After discussion in PR consensus was to just use MAX throughout the
codebase.

ref: rust-bitcoin#4164 (comment)

* Fix amount whole bitcoin constructors

I royally botched the recent effort to make const amount constructors
use a smaller type. I left in an  unnecessary panic and forgot to do
both of them.

Note these function return values will change again very shortly when we
start enforcing the MAX_MONEY invariant. However the 64 to 32 bit change
is unrelated to that and is easier to review if done separately.

Whole bitcoin can not in any sane environment be greater than 21,000,000
which fits in 32 bits so we can take a 32 bit integer in the whole
bitcoin constructors without loss of utility. Doing so removes the
potential panic.

This is a breaking API change. We elect not to deprecate because we want
to keep the same function names.

* Remove panic in dust value functions

Calculating the minimum non-dust fee currently panics if either the
script is really big or the dust fee rate is really big.

Harden the API by returning an `Option` instead of panicing.

* Make mul weight by fee return NumOpResult

Now that we have the `NumOpResult<Amount>` type that is used to show a
math calculation returned a valid amount we can use it when multiplying
weight and fee rates thus removing panics.

* Remove deprecated amount methods

When we enforce the MAX_MONEY invariant these functions would require
the function signature changing - might as well just delete them.

* feat: add MAX_BLOCK_SERIALIZED_SIZE existing in core

* Update ecdsa-psbt.rs

* test: add coverage for ServiceFlags::P2P_V2

* chore: add missing backquotes

Signed-off-by: kevincatty <zhanshanmao@outlook.com>

* Create test helper function to create a header

Move the header creation to the helper function so it can be used in
further tests.

* Increase test coverage in block.rs

Add tests to `primitive::block` module to increase test coverage.

* Use impl_op_for_references macro in fee module

This commit replaces the individual operator implementations in the fee
module with the impl_op_for_references macro to handle reference operations.
This removes the need to manually implement reference combinations for
operands, simplifying the code and improving consistency.

The change:
- Replaces direct implementations of operators with macro usage
- Adds tests to verify that reference operations work correctly
- Maintains the same semantics as the original implementation

* Enforce the MAX_MONEY invariant in amount types

Enforcing the MAX_MONEY invariant is quite involved because it means
multiple things:

- Constructing amounts is now fallible
- Converting from unsigned to signed is now infallible
- Taking the absolute value is now infallible
- Integer overflow is illuminated in various places

Details:

- Update from_sat to check the invariant
- Fix all docs including examples
- Use the unchecked constructor in test code
- Comment any other use of the unchecked constructor
- Deprecate unchecked_abs
- Fail serde (using the horrible string error variant)
- Try not to use the unchecked constructor in rustdocs, no need to encourage unsuspecting users to use it.
- Use ? in rustdoc examples (required by Rust API guidlines)
- Remove TryFrom<Amount> for SignedAmount because the conversion is now infallible. Add a From impl.
- Fix the arbitrary impls
- Maintain correct formatting
- Remove private check_max function as its no longer needed

* Automated update to Github CI to cargo-semver-checks version-0.40.0

* Automated update to Github CI to rustc nightly-2025-03-14

* 2025-03-16 automated rustfmt nightly

* chore: spellchecker

* Update sighash.rs

* Update owned.rs

* Enable getting the network kind from an address

Users may wish to ask of an address 'what kind of address is this?' We
have the `NetworkKind` struct that abstracts over the answer but
currently no API to ask the question.

The address may have been parsed or constructed and weather the network
has been checked already is immaterial. Hence we add the function for
both `NetworkChecked` and `NetworkUnchecked` addresses.

Fix: rust-bitcoin#4247

* Increase test coverage in absolute.rs

Modify existing tests and add new ones to increase test coverage in
`locktime/absolute.rs`.

* Increase test coverage in relative.rs

Increase test coverage in `locktime/relative.rs`

* units: Make from_int_btc_const take a 16 bit integer

The `from_int_btc_const` constructors are specifically designed for
easily creating amount types in const context but currently they return
an error which is annoying to handle in const context. If we make the
`whole_bitcoin` parameter a 16 bit integer this gives us a nicer const
constructor with the downside that it can only create values upto a
maximum of

- unsigned: 65_536
- signed: 32_767

That is plenty high enough for most use cases.

Then use the new `from_int_btc_const` in associated consts.

Note that because `from_sat` checks max (and min) values we must
define max and min from sats directly.

* Add `Buf` suffix to `TaprootMerkleBranch`

This type actually contains a `Vec` but we would prefer to have an
unsized type. Rename it first so that we can reuse the name later.

* Push `merkle_branch` module one level deeper.

This moves the content of the module into `buf` submodule making future
changes clearer.

* Add `as_mut_slice` to `TaprootMerkleBranchBuf`

`TaprootMerkleBranchBuf` already had `as_slice` method and `DerefMut`
but was missing `as_slice_mut`, so this change adds it.

* Introduce unsized `TaprootMerkleBranch`

`TaprootMerkleBranchBuf` being a vec introduced intermediate allocation
when creating or decoding `Witness`. However the representation on the
wire is the same as in-memory (aside from `#[repr(transparent)]`) so
this allocation wasn't really needed.

This commit introduces `TaprootMerkleBranch` type which is unsized and
can be used in place of `TaprootMerkleBranchBuf` within `ControlBlock`.
Aside from removing the intermediate allocation, this improves the API a
bit: the conversion from array to other type is no longer needed because
it's performed by `ControlBlock` in its methods. Thus, consumers who
have an array can simply set it as `merkle_branch` field and then encode
the `ControlBlock` into witness. A convenience method is also provided
to push the `ControlBlock` along with other parts at the end of the
`Witness`.

* Change `Deref::Target` of `TaprootMerkleBranchBuf`

`TaprootMerkleBranchBuf` previously derefed to a slice which lost the
information about length being valid. This commit changes the type
which, while API-breaking, is not disruptive because the type has API
very similar to slice.

* Don't use references to `TaprootMerkleBranchBuf`

The new unsized type is more flexible and so are the references to it.
Just like we pass around `&str` instead of `&String` we should be
passing `&TaprootMerkleBranch` instead of `&TaprootMerkleBranchBuf`.

* amount: rename `from_int_btc_const` unctions to hungarian ones

We have `from_int_btc_const` on both `Amount` and `SignedAmount` because
the "real" `from_int_btc` is generic over the integer type it accepts,
which means that it cannot be a constfn. But we do want a constfn.

However, just because `from_int_btc_const` exists for the sake of
constfn doesn't mean that that's what it *is*. So rename these methods
to reflect what they *are*.

* amount: add from_sat_i32 and from_sat_u32 methods for small constants

We have a ton of calls to `from_sat_unchecked` for small constants which
were clearly in range, e.g. in fee.rs. Add a new constfn for these
cases. Don't bother making a generic Into<u32>/Into<u16> variant because
there isn't an obvious name for it.

There are 7 instances where we're using this method with values that are
out of range, which we leave as from_sat_unchecked for now.

* tests: replace Amount::from_sat_unchecked with from_sat.unwrap

There are only 7 instances of this so just call .unwrap() on each one.

* amount: stop using from_sat_unchecked in tests

There is no need for this. It's a 2-line diff to change it.

* amount: return i64 from parse_signed_to_satoshi

This private function is used for string-parsing an amount. It returns a
sign boolean and a u64, but its surrounding logic can be simplified if
it just returns a i64 (which is OK now since the range of `Amount` fits
into the range of i64).

Along the way we eliminate some instances of from_sat_unchecked.

Annoyingly we still need the bool to distinguish -0 from +0; when
parsing Amount we reject -0 (and we have tests for this).

This causes a couple error constructors to no longer be used outside of
unit tests. May be worth looking at whether we need them, or whether we
should be using them in parse_signed_to_satoshi.

* amount: move MIN/MAX constants and constructors inside the privacy boundary

It's conceptually a bit tortured to have an `Amount` type defined in a
private module, with an _unchecked method allowing you to set values out
of range, which needs to be used outside of the module to *define* the
range and the constructors that check it.

Move the constants and constructors inside the privacy module, where they
can be written directly. This is easier to understand and eliminates a couple
_unchecked calls.

* amount: remove from_sat_unchecked

Turns out we don't even need this. It is only used in one place now and
we can just stick an .expect there.

* Remove a bunch of `try_into().expect()`

Previously we've used `try_into().expect()` because const generics were
unavailable. Then they became available but we didn't realize we could
already convert a bunch of code to not use panicking conversions. But we
can (and could for a while).

This adds an extension trait for arrays to provide basic non-panicking
operations returning arrays, so they can be composed with other
functions accepting arrays without any conversions. It also refactors a
bunch of code to use the non-panicking constructs but it's certainly not
all of it. That could be done later. This just aims at removing the
ugliest offenders and demonstrate the usefulness of this approach.

Aside from this, to avoid a bunch of duplicated work, this refactors
BIP32 key parsing to use a common method where xpub and xpriv are
encoded the same. Not doing this already led to a mistake where xpriv
implemented some additional checks that were missing in xpub. Thus this
change also indirectly fixes that bug.

* Add official BIP32 test vectors for invalid keys

These are defined in the BIP as invalid. The previous commit fixed a bug
where invalid key was parsed as valid and this bug can be caught by
these vectors. Therefore, if this commit is ordered before the last one
the test will fail.

* Use compute_merkle_root

Remove manual implementation of merkle root calculation and just use the
function we already have.

Refactor only, no logic change.

* Remove From<hash> for not-general-hash types

The `hash_newtype` macro is explicitly designed to produce a hash that
is not a general purpose hash type to try and prevent users hashing
arbitrary stuff with it. E.g., `Txid` isn't meant to be just hash
arbitrary data. However we provide a `From` impl that will convert any
instance of the inner hash type into the new type. This kind of defeats
the purpose. We provide `from_byte_array` and `to_byte_array` to allow
folk to 'cast' from one hash type to another if they really want to and
its ugly on purpose.

Also, it is becoming apparent that we may be able to remove the `hashes`
crate from the public API of `primitives` allowing us to stabalise
`primitives` without stabalising `hashes`.

For both these reasons remove the `From` impl from the `hash_newtype`
macro. Note that deprecating doesn't seem to work so we just delete it.

* Remove From<newtype> for $hash

We provide the from/to_byte_array functions for casting between arrays.
We shouldn't be supporting calls to `into` to quickly do the cast.

We already removed the other direction, now remove this one.

* Automated update to Github CI to rustc stable-1.85.1

* Add XOnlyPublicKey support for PSBT key retrieval and improve Taproot signing

This commit enhances PSBT signing functionality by:

1. Added new KeyRequest::XOnlyPubkey variant to support direct retrieval using XOnly public keys
2. Implemented GetKey for HashMap<XOnlyPublicKey, PrivateKey> for more efficient Taproot key management
3. Modified HashMap<PublicKey, PrivateKey> implementation to handle XOnlyPublicKey requests by checking both even and odd parity variants

These changes allow for more flexible key management in Taproot transactions.
Specifically, wallet implementations can now store keys indexed by either
PublicKey or XOnlyPublicKey and successfully sign PSBTs with Taproot inputs.

Added tests for both implementations to verify correct behavior.

Added test for odd parity key retrieval.

Closes rust-bitcoin#4150

* Rename impl_try_from_array to impl_from_array

* Make usage of Self and type uniform across both modules

This commit standardizes the function signatures in the Amount and SignedAmount
implementations by consistently using Self as the return type instead of the concrete
type names. This makes the code more consistent, easier to maintain, and follows Rust's
idiomatic practices.

Changes:

Replace all occurrences of -> Amount with -> Self in unsigned.rs
Replace all occurrences of -> SignedAmount with -> Self in signed.rs
Make similar replacements for Option/Result return types
Use Self:: instead of the explicit type name for static method calls

* Automated update to Github CI to rustc nightly-2025-03-21

* docs: fix LICENCE link

* Add a bunch of missing conversions for `Witness`

`Witness` was missing conversions from arrays (and variations) which was
annoying when creating known-sized witnesses. These come up when
spending statically-known inputs and in tests.

* Impl `PartialEq` between `Witness` and containers

Since `Witness` is semantically equivalent to `&[&[u8]]` they should
also be comparable. However we only had the impl to compare `Witness`
with itself. Being able to compare `Witness` with other containers is
particularly needed in tests.

* Don't access internalls of `Witness` in tests

Accessing the internals of tested object is problematic because it makes
changes to layout harder, it makes tests harder to read and it checks
implementation details rather than semantics of the API (behvaior).

We had such tests in `primitives::witness` so this changes them to use
the newly added APIs instead. However, it still leaves
`from_parts__unstable` which needs to be dealt with separately.

* Simplify `Witness` construction in tests

The `Witness`-related tests were constructing `Witness` in
over-complicated way by serializing `Vec<Vec<u8>>` and then
deserializing `Witness` even though they were not supposed to test
serialization but Taproot accessor methods. This was difficult to
understand and maintain.

This change simplifies them to just construct the `Witness` from array
of `Vec<u8>`s using the recently-added constructors. Note that we
already have serialization tests written separately so we're not losing
meaningful coverage here.

---------

Signed-off-by: costcould <fliter@myyahoo.com>
Signed-off-by: looklose <shishuaiqun@yeah.net>
Signed-off-by: NinaLua <iturf@sina.cn>
Signed-off-by: RiceChuan <lc582041246@gmail.com>
Signed-off-by: kevincatty <zhanshanmao@outlook.com>
Co-authored-by: merge-script <apoelstra@wpsoftware.net>
Co-authored-by: Martin Habovstiak <martin.habovstiak@gmail.com>
Co-authored-by: Tobin C. Harding <me@tobin.cc>
Co-authored-by: Liu-Cheng Xu <xuliuchengxlc@gmail.com>
Co-authored-by: leopardracer <136604165+leopardracer@users.noreply.github.com>
Co-authored-by: costcould <fliter@myyahoo.com>
Co-authored-by: Bruno Garcia <brunoely.gc@gmail.com>
Co-authored-by: jrakibi <j.errakibi@gmail.com>
Co-authored-by: ndungudedan <dnkibere@gmail.com>
Co-authored-by: Erik De Smedt <contact@erikdesmedt.be>
Co-authored-by: leonarddt05 <139609434+leonarddt05@users.noreply.github.com>
Co-authored-by: looklose <shishuaiqun@yeah.net>
Co-authored-by: kilavvy <140459108+kilavvy@users.noreply.github.com>
Co-authored-by: Update Nightly Rustc Bot <bot@example.com>
Co-authored-by: 19年梦醒 <3949379+getong@users.noreply.github.com>
Co-authored-by: Jamil Lambert, PhD <Jamil.Lambert@proton.me>
Co-authored-by: Erick Cestari <erickcestari03@gmail.com>
Co-authored-by: Shing Him Ng <shinghim@protonmail.com>
Co-authored-by: Peter Todd <pete@petertodd.org>
Co-authored-by: NinaLua <iturf@sina.cn>
Co-authored-by: yancy <github@yancy.lol>
Co-authored-by: wgyt <wgythe@gmail.com>
Co-authored-by: RiceChuan <lc582041246@gmail.com>
Co-authored-by: AM <hones02_tunica@icloud.com>
Co-authored-by: ChrisCho-H <c.hyunhum@gmail.com>
Co-authored-by: planetBoy <140164174+Guayaba221@users.noreply.github.com>
Co-authored-by: kevincatty <zhanshanmao@outlook.com>
Co-authored-by: healthyyyoung <healthyoung@proton.me>
Co-authored-by: jike <jike2021@proton.me>
@tcharding tcharding deleted the 03-03-prepare-max-money branch April 14, 2025 04:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

API break This PR requires a version bump for the next release C-bitcoin PRs modifying the bitcoin crate C-units PRs modifying the units crate P-high High priority test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants