Enforce MAX_MONEY invariant in amount types#3794
Enforce MAX_MONEY invariant in amount types#3794tcharding wants to merge 12 commits intorust-bitcoin:masterfrom
Conversation
jamillambert
left a comment
There was a problem hiding this comment.
I somehow feel this is making it all more complicated than it needs to be. Essentially we have changed the max amount to be slightly lower than u64::MAX so that there are not rounding errors in the conversions to f64. When u64::MAX was the maximum it was just accepted that above this means the functions panic and this is fine, and a lot of the tests relied on this too. But can we not just leave it this way with a new arbitrary lower MAX? And just change it so that from_sats panics above MAX_MONEY which is theoretically a value far above any practical amount anyone would want to use, with no unchecked version or any API changes?
No, I don't think this was ever the case. We had this for the arithmetic operators, because in Rust that's just how they work, but aside from those we put quite a bit of effort into avoiding panics (or at least, making them avoidable). |
|
Is it still not possible to leave the functionality the same. i.e what would have paniced above u64::MAX or i64::MAX now panics above MAX_MONEY? And not change the return type of any function or add new functions? |
|
Some things cannot stay the same, no. When we were using |
I think I am starting to understand more what @tcharding means by this. But I don't think backing out of using MAX_MONEY is an option due to the rounding error issues. EDIT: Well that's not entirely true, we could change the other functions that convert to f64 and use e.g. strings instead so that there are no rounding issues and keep |
|
Did a quick pass for now since that's all I have time for a the moment. Plan to look more into it later. |
52339a3 to
3c7af09
Compare
We are trying a new strategy to get to 1.0 more quickly - remove `amount` and `fee` and release everything else. In preparation for release add a changelog entry, bump the version, and update the lock files. `v1.0` here we come. Before we merge this we should add: - rust-bitcoin#3934 or rust-bitcoin#3866 - rust-bitcoin#3933 - rust-bitcoin#3932 - rust-bitcoin#3929 - rust-bitcoin#3926 - rust-bitcoin#3923 - rust-bitcoin#3893 - rust-bitcoin#3866 - rust-bitcoin#3794
We are trying a new strategy to get to 1.0 more quickly - remove `amount` and `fee` and release everything else. In preparation for release add a changelog entry, bump the version, and update the lock files. `v1.0` here we come. Before we merge this we should add: - rust-bitcoin#3934 or rust-bitcoin#3866 - rust-bitcoin#3933 - rust-bitcoin#3932 - rust-bitcoin#3929 - rust-bitcoin#3926 - rust-bitcoin#3923 - rust-bitcoin#3893 - rust-bitcoin#3866 - rust-bitcoin#3794
|
🚨 API BREAKING CHANGE DETECTED To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section. |
Run `just fmt`. No other manual changes.
We have a const for this, use it. Internal change only.
Throughout the `amount::tests` module we use `sat` and `ssat` as aliases to amount constructors but in on test we use them as `Denomination` variables. To assist clarity and so we can introduce uniform usage of the constructor aliases change the variable names to use the `den_` prefix. Internal change only, no logic changes.
|
🚨 API BREAKING CHANGE DETECTED To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section. |
1 similar comment
|
🚨 API BREAKING CHANGE DETECTED To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section. |
|
This is a total re-write - all previous review comments have been marked as resolved. |
|
Come on crew, lets get this done! |
|
🚨 API BREAKING CHANGE DETECTED To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section. |
There is an as yet unresolved discussion about the unchecked amount constructor. In an effort to focus the amount of changes required later and also to make the `tests` module uniform use the `sat` and `ssat` constructor aliases everywhere. Internal change only, no logic changes.
We are about to start enforcing the MAX_MONEY invariant. Doing so will change constructors to return an error type. In preparation use the `_unchecked` constructor for all the consts. Internal change only, no logic changes.
The unchecked-should-be-unsafe conversation is out of scope for this patch. We want to bite off small chunks so the constructors are left as they currently are - we are just doing the encapsulation here. This is in preparation for enforcing the MAX_MONEY invariant which is not currently enforced. As per the sanity rules policy outline in: rust-bitcoin#4090 For both amount types create a private `encapsulate` module that consists of exactly the type and a single constructor and a single getter.
TL;DR For now, just use MAX_MONEY everywhere in this codebase. This git log is long so that it can be discussed and _later_ changed if needed. This is a trivial change, we do not need to bikeshed this at this moment - by picking MAX_MONEY its a simple shell oneliner to change it. Weather we keep both constants or get rid of one is out of scope for this patch. This patch is just about being uniform and suitably expressive. We have two amount constants MAX and MAX_MONEY that are equal. I believe we should pick one and use it throughout the codebase to be uniform. I am open to either option. There are arguments for both sides: (Note please I have explicitly left out the argument "why do we even have two consts" on both sides below for reasons noted above.) Arguments for using MAX: - MAX is more terse - MAX is equally as expressive - Future devs don't need to care about the difference because for them they will have always been the same Arguments for using MAX_MONEY: - MAX_MONEY is more explicit - `::MAX` looks a lot like `u64::MAX` and for Bitcoin devs an amount is intrinsically a 64 byte unsigned integer. - MAX was at one stage in `rust-bitcoin`s life equal to `u64::MAX` and now that has changed so the explicitness might be worth it. [0] I don't personally trust my brain to not mix these up.
I royally botched the recent effort to make const amount constructors use a smaller type. I left in an unnecessary panic and forgot to do both of them. Note these function return values will change again very shortly when we start enforcing the MAX_MONEY invariant. However the 64 to 32 bit change is unrelated to that and is easier to review if done separately. Whole bitcoin can non in any sane environment be greater than 21,000,000 which fits in 32 bits so we can take a 32 bit integer in the whole bitcoin constructors without loss of utility. Doing so removes the potential panic. This is a breaking API change. We elect not to deprecate because we want to keep the same function names.
Calculating the minimum non-dust fee currently panics if either the script is really big or the dust fee rate is really big. Harden the API by returning an `Option` instead of panicing.
Now that we have the `NumOpResult<Amount>` type that is used to show a math calculation returned a valid amount we can use it when multiplying weight and fee rates thus removing panics.
When we enforce the MAX_MONEY invariant these functions would require the function signature changing - might as well just delete them.
Enforcing the MAX_MONEY invariant is quite involved because it means multiple things: - Constructing amounts is now fallible - Converting from unsigned to signed is now infallible - Taking the absolute value is now infallible - Integer overflow is illuminated in various places Details: - Update from_sat to check the invariant - Fix all docs including examples - Use the unchecked constructor in test code - Comment any other use of the unchecked constructor - Deprecate unchecked_abs - Fail serde (using the horrible string error variant) - Try not to use the unchecked constructor in rustdocs, no need to encourage unsuspecting users to use it. - Use ? in rustdoc examples (required by Rust API guidlines) - Remove TryFrom<Amount> for SignedAmount because the conversion is now infallible. Add a From impl. - Fix the arbitrary impls - Maintain correct formatting - Remove private check_max function as its no longer needed
|
🚨 API BREAKING CHANGE DETECTED To see the changes click details on "Check semver breaks / PR Semver - stable toolchain" job then expand "Run semver checker script" and scroll to the end of the section. |
|
Damn, Friday afternoon and I didn't quite make it over the line TODO:
|
Maybe it would be helpful to open a new PR if it's a re-write. There's a lot of scar tissue on this one. |
|
Replaced by #4157 |
Enforcing the MAX_MONEY invariant is quite involved because it means multiple things:
Enforcing the
MAX_MONEYinvariant is quite involved because it means multiple things:Details:
from_satto check the invariantunchecked_abs?in rustdoc examples (required by Rust API guidlines)TryFrom<Amount> for SignedAmountbecause the conversion is now infallible. Add aFromimpl.check_maxfunction as its no longer neededClose #620