Skip to content

Enforce MAX_MONEY invariant in amount types#3794

Closed
tcharding wants to merge 12 commits intorust-bitcoin:masterfrom
tcharding:12-20-amount
Closed

Enforce MAX_MONEY invariant in amount types#3794
tcharding wants to merge 12 commits intorust-bitcoin:masterfrom
tcharding:12-20-amount

Conversation

@tcharding
Copy link
Copy Markdown
Member

@tcharding tcharding commented Dec 19, 2024

Enforcing the MAX_MONEY invariant is quite involved because it means multiple things:

Enforcing the MAX_MONEY invariant is quite involved because it means multiple things:

  • Constructing amounts is now fallible
  • Converting from unsigned to signed is now infallible
  • Taking the absolute value is now infallible
  • Integer overflow is illuminated in various places

Details:

  • Update from_sat to check the invariant
  • Fix all docs including examples
  • Use the unchecked constructor in test code
  • Comment any other use of the unchecked constructor
  • Deprecate unchecked_abs
  • Fail serde (using the horrible string error variant)
  • Try not to use the unchecked constructor in rustdocs, no need to encourage unsuspecting users to use it.
  • Use ? in rustdoc examples (required by Rust API guidlines)
  • Remove TryFrom<Amount> for SignedAmount because the conversion is now infallible. Add a From impl.
  • Fix the arbitrary impls
  • Maintain correct formatting
  • Remove private check_max function as its no longer needed

Close #620

@github-actions github-actions bot added C-bitcoin PRs modifying the bitcoin crate C-units PRs modifying the units crate C-internals PRs modifying the internals crate test C-primitives labels Dec 19, 2024
@tcharding tcharding marked this pull request as draft December 19, 2024 23:44
Copy link
Copy Markdown
Contributor

@jamillambert jamillambert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I somehow feel this is making it all more complicated than it needs to be. Essentially we have changed the max amount to be slightly lower than u64::MAX so that there are not rounding errors in the conversions to f64. When u64::MAX was the maximum it was just accepted that above this means the functions panic and this is fine, and a lot of the tests relied on this too. But can we not just leave it this way with a new arbitrary lower MAX? And just change it so that from_sats panics above MAX_MONEY which is theoretically a value far above any practical amount anyone would want to use, with no unchecked version or any API changes?

@apoelstra
Copy link
Copy Markdown
Member

When u64::MAX was the maximum it was just accepted that above this means the functions panic and this is fine

No, I don't think this was ever the case. We had this for the arithmetic operators, because in Rust that's just how they work, but aside from those we put quite a bit of effort into avoiding panics (or at least, making them avoidable).

@jamillambert
Copy link
Copy Markdown
Contributor

Is it still not possible to leave the functionality the same. i.e what would have paniced above u64::MAX or i64::MAX now panics above MAX_MONEY? And not change the return type of any function or add new functions?

@apoelstra
Copy link
Copy Markdown
Member

Some things cannot stay the same, no. When we were using u64::MAX this meant that we could always infallibly convert from a u64 to an Amount. Now we can not. (And conversely, before we couldn't infallibly convert Amount to SignedAmount, but now we can.)

@yancyribbens yancyribbens mentioned this pull request Dec 20, 2024
@jamillambert
Copy link
Copy Markdown
Contributor

jamillambert commented Dec 20, 2024

This is a non-trivial change.

I think I am starting to understand more what @tcharding means by this. But I don't think backing out of using MAX_MONEY is an option due to the rounding error issues.

EDIT: Well that's not entirely true, we could change the other functions that convert to f64 and use e.g. strings instead so that there are no rounding issues and keep u64::MAX as the max.

@yancyribbens
Copy link
Copy Markdown
Contributor

Did a quick pass for now since that's all I have time for a the moment. Plan to look more into it later.

tcharding added a commit to tcharding/rust-bitcoin that referenced this pull request Feb 24, 2025
We are trying a new strategy to get to 1.0 more quickly - remove
`amount` and `fee` and release everything else.

In preparation for release add a changelog entry, bump the version, and
update the lock files.

`v1.0` here we come.

Before we merge this we should add:

- rust-bitcoin#3934 or rust-bitcoin#3866
- rust-bitcoin#3933
- rust-bitcoin#3932
- rust-bitcoin#3929
- rust-bitcoin#3926
- rust-bitcoin#3923
- rust-bitcoin#3893
- rust-bitcoin#3866
- rust-bitcoin#3794
tcharding added a commit to tcharding/rust-bitcoin that referenced this pull request Feb 24, 2025
We are trying a new strategy to get to 1.0 more quickly - remove
`amount` and `fee` and release everything else.

In preparation for release add a changelog entry, bump the version, and
update the lock files.

`v1.0` here we come.

Before we merge this we should add:

- rust-bitcoin#3934 or rust-bitcoin#3866
- rust-bitcoin#3933
- rust-bitcoin#3932
- rust-bitcoin#3929
- rust-bitcoin#3926
- rust-bitcoin#3923
- rust-bitcoin#3893
- rust-bitcoin#3866
- rust-bitcoin#3794
@github-actions
Copy link
Copy Markdown

🚨 API BREAKING CHANGE DETECTED

To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section.

Run `just fmt`. No other manual changes.
We have a const for this, use it.

Internal change only.
Throughout the `amount::tests` module we use `sat` and `ssat` as aliases
to amount constructors but in on test we use them as `Denomination`
variables. To assist clarity and so we can introduce uniform usage of
the constructor aliases change the variable names to use the `den_`
prefix.

Internal change only, no logic changes.
@github-actions
Copy link
Copy Markdown

🚨 API BREAKING CHANGE DETECTED

To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section.

1 similar comment
@github-actions
Copy link
Copy Markdown

🚨 API BREAKING CHANGE DETECTED

To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section.

@tcharding
Copy link
Copy Markdown
Member Author

This is a total re-write - all previous review comments have been marked as resolved.

@tcharding
Copy link
Copy Markdown
Member Author

Come on crew, lets get this done!

@github-actions
Copy link
Copy Markdown

🚨 API BREAKING CHANGE DETECTED

To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section.

There is an as yet unresolved discussion about the unchecked amount
constructor. In an effort to focus the amount of changes required later
and also to make the `tests` module uniform use the `sat` and `ssat`
constructor aliases everywhere.

Internal change only, no logic changes.
We are about to start enforcing the MAX_MONEY invariant. Doing so will
change constructors to return an error type.

In preparation use the `_unchecked` constructor for all the consts.

Internal change only, no logic changes.
The unchecked-should-be-unsafe conversation is out of scope for this
patch. We want to bite off small chunks so the constructors are left as
they currently are - we are just doing the encapsulation here. This is
in preparation for enforcing the MAX_MONEY invariant which is not
currently enforced.

As per the sanity rules policy outline in:

 rust-bitcoin#4090

For both amount types create a private `encapsulate` module that
consists of exactly the type and a single constructor and a single
getter.
TL;DR For now, just use MAX_MONEY everywhere in this codebase.

This git log is long so that it can be discussed and _later_ changed if
needed. This is a trivial change, we do not need to bikeshed this at
this moment - by picking MAX_MONEY its a simple shell oneliner to change
it.

Weather we keep both constants or get rid of one is out of scope for
this patch. This patch is just about being uniform and suitably
expressive.

We have two amount constants MAX and MAX_MONEY that are equal. I believe
we should pick one and use it throughout the codebase to be uniform. I
am open to either option. There are arguments for both sides:

(Note please I have explicitly left out the argument "why do we even
have two consts" on both sides below for reasons noted above.)

Arguments for using MAX:

- MAX is more terse
- MAX is equally as expressive
- Future devs don't need to care about the difference because
  for them they will have always been the same

Arguments for using MAX_MONEY:

- MAX_MONEY is more explicit
- `::MAX` looks a lot like `u64::MAX` and for Bitcoin devs an amount is
  intrinsically a 64 byte unsigned integer.
- MAX was at one stage in `rust-bitcoin`s life equal to `u64::MAX` and
  now that has changed so the explicitness might be worth it.

[0] I don't personally trust my brain to not mix these up.
I royally botched the recent effort to make const amount constructors
use a smaller type. I left in an  unnecessary panic and forgot to do
both of them.

Note these function return values will change again very shortly when we
start enforcing the MAX_MONEY invariant. However the 64 to 32 bit change
is unrelated to that and is easier to review if done separately.

Whole bitcoin can non in any sane environment be greater than 21,000,000
which fits in 32 bits so we can take a 32 bit integer in the whole
bitcoin constructors without loss of utility. Doing so removes the
potential panic.

This is a breaking API change. We elect not to deprecate because we want
to keep the same function names.
Calculating the minimum non-dust fee currently panics if either the
script is really big or the dust fee rate is really big.

Harden the API by returning an `Option` instead of panicing.
Now that we have the `NumOpResult<Amount>` type that is used to show a
math calculation returned a valid amount we can use it when multiplying
weight and fee rates thus removing panics.
When we enforce the MAX_MONEY invariant these functions would require
the function signature changing - might as well just delete them.
Enforcing the MAX_MONEY invariant is quite involved because it means
multiple things:

- Constructing amounts is now fallible
- Converting from unsigned to signed is now infallible
- Taking the absolute value is now infallible
- Integer overflow is illuminated in various places

Details:

- Update from_sat to check the invariant
- Fix all docs including examples
- Use the unchecked constructor in test code
- Comment any other use of the unchecked constructor
- Deprecate unchecked_abs
- Fail serde (using the horrible string error variant)
- Try not to use the unchecked constructor in rustdocs, no need to encourage unsuspecting users to use it.
- Use ? in rustdoc examples (required by Rust API guidlines)
- Remove TryFrom<Amount> for SignedAmount because the conversion is now infallible. Add a From impl.
- Fix the arbitrary impls
- Maintain correct formatting
- Remove private check_max function as its no longer needed
@github-actions
Copy link
Copy Markdown

🚨 API BREAKING CHANGE DETECTED

To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section.

@tcharding
Copy link
Copy Markdown
Member Author

tcharding commented Feb 28, 2025

Damn, Friday afternoon and I didn't quite make it over the line

TODO:

  • Fix kani stuff
  • Fix test in bitcoin/tests (I wish cargo test --color=always --workspace --all-features --all-targets would run all the tests so I didn't get bitten by that all the time EDIT: its --all-features that was hiding the issue).

@yancyribbens
Copy link
Copy Markdown
Contributor

Total re-write.

Maybe it would be helpful to open a new PR if it's a re-write. There's a lot of scar tissue on this one.

@tcharding
Copy link
Copy Markdown
Member Author

Replaced by #4157

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

API break This PR requires a version bump for the next release C-bitcoin PRs modifying the bitcoin crate C-primitives C-units PRs modifying the units crate P-high High priority test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

units: Improve Amount type with guarantees on value being in range

5 participants