-
Notifications
You must be signed in to change notification settings - Fork 38.7k
fuzz: Rework strong and weak net enum fuzzing #20789
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
faac011 to
fa9ab9a
Compare
|
Concept ACK Nice improvements! |
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
vasild
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK fa9ab9a8c
The chance of 5 random commits to all start with the same two letters is 0.00000002% 👀 😮
src/test/util/net.h
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if it is possible to get a warning if a new enum value is added, but this array is forgotten to be updated? I think not :/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is ok as it is, and I am just wondering if this can be improved wrt minimizing the chance of the actual enum and the "all" arrays going out of sync over time. Maybe one of the below would help:
o Add a comment next to the enum definition "don't forget to update ALL_... in src/test/util/net.h if you change this enum".
o Define the array immediately below the enum itself.
o Define both in one struct
struct Color
{
enum E : uint8_t {
RED,
GREEN,
BLUE,
};
static constexpr E all[]{
RED,
GREEN,
BLUE,
};
};
...
std::cout << Color::BLUE << std::endl;
for (auto& c : Color::all) {
std::cout << c << std::endl;
}There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps we could start using kMaxValue as in the Chromium project. That way we could assert that the last element of the ALL_* array is equal to the enum's kMaxValue.
Another benefit from using kMaxValue is that we could start using FuzzedDataProvider::ConsumeEnum when fuzzing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, that is another way. enum Network uses this. It only works as long as enum values are left at their defaults:
Line 38 in f1f26b8
| * Keep these sequential starting from 0 and `NET_MAX` as the last entry. |
and also the dummy "max" value has to be explicitly handled in switch without default:
Lines 145 to 146 in f1f26b8
| case NET_MAX: | |
| assert(false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For strong enums, I was thinking about a macro ENUM_CLASS_ALL that writes the enum class and std::array that holds all enum types of that class. Thus, we could keep using PickValueInArray for strong enums. FuzzedDataProvider::ConsumeEnum is an alternative for strong enums that does exactly the same from the perspective of the fuzz engine.
For weak enums, ConsumeWeakEnum is probably the best bet. I tried kMaxValue and it performed strictly worse in all scenarios, in one it was unable to find a crash at all. And it can't possibly work for 64-bit enums. Someone should report those bugs to google. See also the benchmark: #20789 (comment)
fa9ab9a to
fadbf6a
Compare
|
Force pushed to fix a bug. Should be trivial to re-ACK |
vasild
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK fa
|
cr ACK fadbf6a83040b7a8b9f91e27b61adfaf4c5df3fb Thanks for providing benchmarks! |
|
@vasild btw for the merge script to pick up your ack, you'll have to provide at least the first 6 chars of the git hash ;) |
fadbf6a to
fac2b6c
Compare
|
ACK fac2b6c Both fadbf6a83 and fac2b6c compile fine with clang 12.0.0. @MarcoFalke so you were lucky enough to get 15 straight commit ids all starting with Happy New Year! 🎉 |
|
ACK fac2b6c Code looks good to me, built locally and did some testing, as in the benchmarks. As for the worse performance of kMaxValue, I think calling this a "bug" that should be reported is too hard: |
|
Oh, this is missing from the documentation in the declaration: bitcoin/src/test/fuzz/FuzzedDataProvider.h Lines 70 to 71 in e75f91e
It is only mentioned in the implementation: bitcoin/src/test/fuzz/FuzzedDataProvider.h Lines 291 to 298 in e75f91e
|
fac2b6c to
fa58fe3
Compare
|
Rebased due to trivial conflict in adjacent line |
|
Concept ACK, first read-through looks very good. |
fa58fe3 to
eeee43b
Compare
|
Rebased after removal of my_starting_height |
|
Btw, should be (relatively) easy to re-ACK 🙏 |
|
ACK eeee43b after rebase. |









The fuzz tests have several problems:
net_permission_flagsis outdatedFix all issues in the commits in this pull