Merged
Conversation
The elfutils project was integrated into OSS-Fuzz in google#6670 where Dockerfile pointed to a fork of the official repository with a series of patches that were supposed to make it compile on OSS-Fuzz. Apart from that there was a fuzz target that effectively wrapped the readelf utility by applying a patch to its source code. On the whole it worked at the time but I think there are a few issues: 1. It's hard to point OSS-Fuzz to the official repository (because most of the patches touch the build system and they can't always be applied cleanly); 2. It's almost impossible to add new fuzz targets covering other use cases; 3. It's not possible to build fuzz targets without Docker 4. Since the fuzz target mostly wraps the readelf utility it looks more like a CLI tool than a fuzz target. It calls exit when it should just return 0 to let it keep going and so on. This PR should addresses all those issues apart from 4. The fuzz target was just removed and another one was added instead. (It can be added later though but since it isn't exactly maintainable with the build script pointing at the official repository it should probably be rewritten: https://sourceware.org/pipermail/elfutils-devel/2021q4/004295.html) The new fuzz target covers the code that `systemd` uses to parse untrusted data. Currently it can be used to trigger various issues like heap-buffer-overflows and inifinite loops that in theory can bring down coredump processing on machines where systemd-coredump is used by default. Even though those issues were discovered by one of `systemd` fuzz targets I think elfutils bugs should be caught and reported by elfutils fuzz targets.
Collaborator
|
Thanks! The removal of the readelf fuzzer is that because of (4) or because of the the build script pointing to a wrong repo? If it's because of (4) then FYI in the binutils project we do this for a lot of the binutils utilities, including readelf: https://github.com/google/oss-fuzz/blob/master/projects/binutils/fuzz_readelf.c In Binutils, the consequences have been that we do run into exits and asserts, but the fuzzer have still found lots of issues. The same pattern applies to the other utilities in binutils |
Contributor
Author
It's partly because of (4) but for the most part I just don't think it can be maintained when the build script points to the master branch of the official repository (which keeps rolling forward and tends to make patches like that irrelevant). It's possible to bring it back but I can say that I'm not going to maintain it :-) If someone else is willing to fix it every time it breaks I can of course consider bringing it back. |
DavidKorczynski
approved these changes
Dec 1, 2021
Collaborator
I agree! I might take a look at it a bit later since I have done a fair share on the binutils project and would assume some work can be reused. |
Contributor
Author
|
As far as I know, unlike |
Collaborator
|
Interesting! With that said during ~18 months in existence for the binutils readelf fuzzer I believe there has been 0-2 breakings due to the internals of readelf being changed. I.e. the |
Contributor
Author
|
Looking at the bug reports the old fuzz target has found in about a month, it seems all of them have something to do with the "readelf" utility itself. They should probably be fixed eventually but I'm not sure memory-leaks in short-lived CLI tools or more or less expected calls to "exit" while parsing malformed ELF-files can be considered a priority. @izzeem I wonder if you have reported them to the maintainers of elfutils? If not I can probably report (or maybe fix) them once I've gotten round to it. |
Contributor
|
Hey @evverx. I havent reported any of the |
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Dec 19, 2021
[v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Dec 19, 2021
[v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is tested. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Dec 21, 2021
[v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Dec 21, 2021
[v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Dec 21, 2021
[v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Dec 26, 2021
[v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Dec 26, 2021
[v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Jan 4, 2022
[v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Jan 4, 2022
[v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Jan 8, 2022
[v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Jan 8, 2022
[v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Mar 21, 2022
[v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Mar 21, 2022
[v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Mar 21, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were added. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Mar 21, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were added. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Mar 22, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Mar 23, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Mar 25, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Mar 30, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Apr 3, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Apr 5, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Apr 23, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
evverx
added a commit
to evverx/elfutils
that referenced
this pull request
Apr 25, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
MartinPetkov
pushed a commit
to MartinPetkov/oss-fuzz
that referenced
this pull request
Aug 15, 2022
The elfutils project was integrated into OSS-Fuzz in google#6670 where Dockerfile pointed to a fork of the official repository with a series of patches that were supposed to make it compile on OSS-Fuzz. Apart from that there was a fuzz target that effectively wrapped the readelf utility by applying a patch to its source code. On the whole it worked at the time but I think there are a few issues: 1. It's hard to point OSS-Fuzz to the official repository (because most of the patches touch the build system and they can't always be applied cleanly); 2. It's almost impossible to add new fuzz targets covering other use cases; 3. It's not possible to build fuzz targets without Docker 4. Since the fuzz target mostly wraps the readelf utility it looks more like a CLI tool than a fuzz target. It calls exit when it should just return 0 to let it keep going and so on. This PR should addresses all those issues apart from 4. The fuzz target was just removed and another one was added instead. (It can be added later though but since it isn't exactly maintainable with the build script pointing at the official repository it should probably be rewritten: https://sourceware.org/pipermail/elfutils-devel/2021q4/004295.html) The new fuzz target covers the code that `systemd` uses to parse untrusted data. Currently it can be used to trigger various issues like heap-buffer-overflows and inifinite loops that in theory can bring down coredump processing on machines where systemd-coredump is used by default. Even though those issues were discovered by one of `systemd` fuzz targets I think elfutils bugs should be caught and reported by elfutils fuzz targets.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The elfutils project was integrated into OSS-Fuzz in
#6670 where
Dockerfile pointed to a fork of the official repository
with a series of patches that were supposed to make it compile
on OSS-Fuzz. Apart from that there was a fuzz target that
effectively wrapped the readelf utility by applying a patch
to its source code. On the whole it worked at the time
but I think there are a few issues:
It's hard to point OSS-Fuzz to the official repository
(because most of the patches touch the build system and
they can't always be applied cleanly);
It's almost impossible to add new fuzz targets covering
other use cases;
It's not possible to build fuzz targets without Docker
Since the fuzz target mostly wraps the readelf utility
it looks more like a CLI tool than a fuzz target. It calls
exit when it should just return 0 to let it keep going
and so on.
This PR should address all those issues apart from 4. The fuzz
target was just removed and another one was added instead. (It can
be added later though but since it isn't exactly maintainable with
the build script pointing at the official repository it should
probably be rewritten:
https://sourceware.org/pipermail/elfutils-devel/2021q4/004295.html)
The new fuzz target covers the code that
systemduses to parseuntrusted data. Currently it can be used to trigger various issues
like heap-buffer-overflows and inifinite loops that in theory can bring down
coredump processing on machines where systemd-coredump is used by
default. Even though those issues were discovered by one of
systemdfuzz targets I think elfutils bugs should be caught and reported
by elfutils fuzz targets.
cc @fche @izzeem