pow: add the API to calculate next pow#2090
pow: add the API to calculate next pow#2090vincenzopalazzo wants to merge 1 commit intorust-bitcoin:masterfrom
Conversation
junderw
left a comment
There was a problem hiding this comment.
The calculations in this function are unrelated to the calculation of a new target.
Please fix the calculations, and if possible the spelling nits too.
c9976c8 to
ad69ac5
Compare
|
I've rebased and pushed the changes that I understood from the review. I've also added the testnet work as future work, as discussed with @apoelstra. Regarding some of the feedback that @junderw provided, I couldn't find a solution. I was only told that my approach was incorrect, but I'm unsure how to modify my code to address the identified issues. Could you please provide a clearer review to guide me in fulfilling your requirements? Thank you. |
ad69ac5 to
ec888a9
Compare
The requirements aren't special. Just "perform correct calculations." I don't understand how much clearer I can be without re-writing your code completely. Adding tests might help prove your calculations are right. You can even take some values from mainnet blocks and plug them in and make sure you get the same results. |
|
Here's a single test to help you. Currently it fails #[test]
fn test_calculate_next_work_required() {
use std::str::FromStr;
let mainnet_params = Params::new(crate::Network::Bitcoin);
// Difficulty 808416 -> 810432 on mainnet
// Expected result bits = 0x1704e90f (bits of 810432)
let expected = 0x1704e90f;
// We take the timestamp of 810431 (the block before)
let height_810431 = Height::from_consensus(810431).unwrap();
let timestamp_810431 = 0x651bc919;
// This closure should return the header for 808416 since 810431 - 2015 is 808416
let fetch_header_808416 = |_height| {
// Header for 808416
Ok(Header {
version: crate::block::Version::from_consensus(0x29322000),
prev_blockhash: BlockHash::from_str(
"000000000000000000027ecc78c2da1cc5c0b0496706baa7e4d7c80812c10bf3",
)
.unwrap(),
merkle_root: crate::TxMerkleNode::from_str(
"b920d5b5ebef4e9d106072944e0729cea8bf6defc583a7d87063041a316a757b",
)
.unwrap(),
time: 0x650964b5,
bits: CompactTarget(0x1704ed7f),
nonce: 0xc637a163,
})
};
let result = calculate_next_work_required(
mainnet_params,
height_810431,
timestamp_810431,
fetch_header_808416,
)
.unwrap();
assert_eq!(result, expected);
} |
|
Please see vincenzopalazzo#1 for fixes. |
|
Found a bug in the Work constants #2103 |
You are right, I get completely confused! now review your code I get where I was going. Thanks for doing that |
0a5e854 to
a6941ae
Compare
a6941ae to
754e481
Compare
tcharding
left a comment
There was a problem hiding this comment.
I did not review the logic in the calculate_next_work_required function, to do so requires me to understand the Core code since this mirrors it. It would be really nice if that was not the case, eg. what do the div by 4 and mul by 4s mean?
You can ignore me if you like, but we will need to get 2 reviewers to understand the logic.
0471c86 to
adba1ed
Compare
|
Hey mate, I spent the afternoon playing with this PR. The only way I understand hard things is to code them myself so I re-wrote a new buggy version. I can write a review here but it is probably easier for you to just take a look and grab anything you like and leave out anything you don't. Then I'll re-review again. EDIT: Oh sick, while checking the link I saw the bug! |
bitcoin/src/pow.rs
Outdated
There was a problem hiding this comment.
I think many projects will desire async support here, can we get rid of the closure or something? IIUC this is just pulling all the blocks from the retarget period so we could just accept IntoIterator<Item=I>, I: Borrow<Header>. People would then just fetch all the blocks first and compute the difficulty afterwards. (I know it's a bit wasteful but I don't have a better idea that doesn't involve confusing API workarounds.) This also makes tests much easier.
Also if we do end up with closure we should use Height in the parameter.
There was a problem hiding this comment.
We don't need all the blocks; we only need the starting and ending blocks.
There was a problem hiding this comment.
So I guess there's an even easier soluton where we just take those headers directly as arguments :).
There was a problem hiding this comment.
If we only take the first and last block [header] we put the burden of doing this loop on the user
https://github.com/bitcoin/bitcoin/blob/82ea4e787c791acbc85fd3043dd6bae038cba4f2/src/pow.cpp#L32
FTR I don't actually understand what the checks are for in this loop.
There was a problem hiding this comment.
(And they have to understand that this loop is only required for testnet, so no one is going to do it.)
There was a problem hiding this comment.
I think many projects will desire async support here, can we get rid of the closure or something?
Does the async support require also an async caller? otherwise, it is just a blocking call, right?
But I also like the idea of passing it as an input parameter, and the @tcharding is a good point.
This is the era of choice, we must support async or we do not want it at the API level right now? I leave you the fun of choose 😄 because I am bias
bitcoin/src/pow.rs
Outdated
There was a problem hiding this comment.
I'd put this inside if to more obviously avoid costly computation.
bitcoin/src/pow.rs
Outdated
There was a problem hiding this comment.
Crap, this is confusing, looks like duplicated line! :D (there's no solution, I'm just ranting)
There was a problem hiding this comment.
ha, I tried twice to make this nicer and couldn't come up with anything.
bitcoin/src/pow.rs
Outdated
There was a problem hiding this comment.
OT: we really should have all_zeros as an inherent method. I think I did that in split hash triat PR.
bitcoin/src/pow.rs
Outdated
There was a problem hiding this comment.
You're converting everything to consensus and then comparing. You could (and most likely should) just not convert and compare the strong types instead. In fact, if there's any reason not to something is probably horribly wrong.
There was a problem hiding this comment.
I don't think we have comparison opcodes on CompactTarget. If we do we are planning to get rid of them. See #2110. (I see that you commented there that we should have explict named methods for the different comparison methods. Ok, but they don't exist right now.)
There was a problem hiding this comment.
FWIW, the pow stuff is on the top of my list at the moment, I started on it yesterday.
There was a problem hiding this comment.
Oh, crap, so we have an actual use case for comparison, kinda undermining the point of #2110, but it's still obscure. It's also super weird that Core compares those. Makes me quite intrigued to figure out WTF is going on.
Using Eq now could be a nice way to not forget to change the code (presumably to use explicit method) when #2110 is resolved.
There was a problem hiding this comment.
Well, yes, we have a use case within PoW computation. But there's lots of stuff we do in that computation that we don't necessarily want to expose by itself.
|
@Kixunil suggested new functions |
|
copied from lightningdevkit/rust-lightning#2124 (comment)
I don't see any force pushes in the last 5 days, is the github UI glitching? |
What is wrong with being like C++? I do not see anything wrong with C++ style. However, the bad thing about the C++ version is that there are more than 4 people trying to understand the core code, so the only thing that would be good to improve is the documentation and the code, other black magic rust is just making the core difficult to read when the time to say "I wrote yesterday still looks a lot like Rust." will come. I will go through the review over the weekend |
There's a reason we use quite different style in Rust. Aside from many bugs, Rust is much more readable. Min, max a and clamp are not black magic. |
This is also possible in C++ with Other than using the |
Sorry @tcharding I got overwhelmed by the rust-bitcoin notification and I lost it inside my notification view. You should see the force push now |
176b249 to
a0cb47b
Compare
|
I, and likely most Rustceans, find those methods more readable. That's a good enough reason already. But it's not something I would block a PR for. But it's possible we'll have to change it anyway if a future version of clippy has a lint for this. |
|
|
Oh, it has MSRV 1.50, sorry about that. Skip that one and consider adding |
55828ad to
c0785a2
Compare
This is a initial work to support a helper function to calculate a next pow inside rust-bitcoin itself. Link: rust-bitcoin#1997 Co-Developed-by: @junderw Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
c0785a2 to
3f05d67
Compare
Who is the second "we" referring to? After spending time on this last week I am more than comfortable to say I understand the core code and can re-write it in Rust style.
I can take ownership of this, I was intentionally not doing so because I didn't want to steam roll over you since you got here first. If you don't mind how this gets done and just want to see the functionality in If there is any other functionality missing that nakamoto needs please do say so, I'm working me hardest to support folk these days. I have not personally looked at the nakamoto codebase though. To clarify, if you leave this open I'll assume you want to do the work and will review and wait for you. If you close this PR I'll push forward with #2180. |
|
I think we should close in favor of 2180. I am willing to review this PR again though, if it needs reviewing. |
|
I think the sync API is much more concerning than any code style around min/max. |
|
@tcharding up to you, I really do not care who is the author of the PR or how the code will be written more rusty or not Cpp style. I care just that if I have some bugs the code is clear that I can read it without cry The code that will be merged will have no API user, so we do not know if with this API I will be able to sync nakamoto with the network and fully validate the chain or if the user of the API needs some different UX. So my priority is to have this functionality so I can continue to do my work and provide more tests about the code. When we have this code merged then I need to way the wallet user to update to the new version of rust bitcoin and then I can test nakamoto. So this sounds a work that will take months.
We can also think of moving the function in an async style due to that in the rust compiler we are doing some work to make the async programming usable. However, this is not good for people who are using not standard async IO (e.g nakamoto) or low-level API with mio and I do not like forcing this async style without having some concrete user. e.g: If nakamoto is going to use this API we do not need an async function |
|
Maybe we can modify the API to be completion based? I know it can have performance issues but if it's done right it might be fine. Like what I did with |
Can you close this if you are ok for me to work on #2180, I don't want to close your PR.
Don't worry, writing code that is brain-dead simple to read is my thing :) |
This is an initial work to support a helper function to calculate the next pow inside rust-bitcoin itself.
Fixes: #1997
This is out for early feedback on API design and also possible improvement that I did not see in the algorithm implementation