Skip to content

Commit 17eccb3

Browse files
shinghimshinghim
committed
Migrate from Honggfuzz to libfuzzer
This commit has a few main parts: - Update CI to install required dependencies for libfuzzer The default Cross s390x image uses an older version of GCC (Ubuntu 20.04), which lacks std::clamp needed by libfuzzer-sys. The s390x Dockerfile provides Cross with a container with GCC 13, native build tools, and QEMU for running s390x tests. Cross.toml configures Cross to use this custom image instead of the default. - Update the lock files - Update the fuzz targets to use libfuzzer - Update the fuzz helper scripts and documentation - Update the duplicate dependency whitelist to include libc - Move `fuzz` into a nested workspace
1 parent dcd414e commit 17eccb3

45 files changed

Lines changed: 612 additions & 1207 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/docker/Dockerfile.cross‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
FROM ubuntu:24.04
2+
3+
RUN apt-get update && \
4+
apt-get install -y --no-install-recommends \
5+
gcc g++ \
6+
gcc-14-s390x-linux-gnu \
7+
g++-14-s390x-linux-gnu \
8+
libc6-dev-s390x-cross \
9+
libstdc++-14-dev-s390x-cross \
10+
gcc-14-aarch64-linux-gnu \
11+
g++-14-aarch64-linux-gnu \
12+
libc6-dev-arm64-cross \
13+
libstdc++-14-dev-arm64-cross \
14+
qemu-user-static \
15+
binfmt-support \
16+
curl \
17+
ca-certificates && \
18+
rm -rf /var/lib/apt/lists/*
19+
20+
RUN update-alternatives --install /usr/bin/s390x-linux-gnu-gcc s390x-linux-gnu-gcc /usr/bin/s390x-linux-gnu-gcc-14 100 && \
21+
update-alternatives --install /usr/bin/s390x-linux-gnu-g++ s390x-linux-gnu-g++ /usr/bin/s390x-linux-gnu-g++-14 100
22+
23+
RUN update-alternatives --install /usr/bin/aarch64-linux-gnu-gcc aarch64-linux-gnu-gcc /usr/bin/aarch64-linux-gnu-gcc-14 100 && \
24+
update-alternatives --install /usr/bin/aarch64-linux-gnu-g++ aarch64-linux-gnu-g++ /usr/bin/aarch64-linux-gnu-g++-14 100
25+
26+
RUN update-binfmts --enable qemu-s390x || true
27+
28+
RUN update-binfmts --enable qemu-aarch64 || true
29+
30+
RUN gcc --version && \
31+
s390x-linux-gnu-gcc --version && \
32+
s390x-linux-gnu-g++ --version
33+
34+
RUN aarch64-linux-gnu-gcc --version && \
35+
aarch64-linux-gnu-g++ --version
36+
37+
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal
38+
ENV PATH="/root/.cargo/bin:${PATH}"
39+
RUN rustup target add s390x-unknown-linux-gnu
40+
41+
RUN rustup target add aarch64-unknown-linux-gnu
42+
43+
ENV CARGO_TARGET_S390X_UNKNOWN_LINUX_GNU_LINKER=s390x-linux-gnu-gcc \
44+
CARGO_TARGET_S390X_UNKNOWN_LINUX_GNU_RUNNER="qemu-s390x-static -L /usr/s390x-linux-gnu" \
45+
CC_s390x_unknown_linux_gnu=s390x-linux-gnu-gcc \
46+
CXX_s390x_unknown_linux_gnu=s390x-linux-gnu-g++ \
47+
CXXFLAGS_s390x_unknown_linux_gnu="-std=c++17"
48+
49+
ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \
50+
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_RUNNER="qemu-aarch64-static -L /usr/aarch64-linux-gnu" \
51+
CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc \
52+
CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++ \
53+
CXXFLAGS_aarch64_unknown_linux_gnu="-std=c++17"

‎.github/workflows/cron-daily-fuzz.yml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,6 @@ jobs:
5959
with:
6060
persist-credentials: false
6161
- uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
62-
6362
id: cache-fuzz
6463
with:
6564
path: |
@@ -97,4 +96,4 @@ jobs:
9796
- name: Display structure of downloaded files
9897
run: ls -R
9998
- run: find executed_* -type f -exec cat {} + | sort > executed
100-
- run: source ./fuzz/fuzz-util.sh && listTargetNames | sort | diff - executed
99+
- run: cargo fuzz list | sort | diff - executed

‎.github/workflows/rust.yml‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -262,8 +262,15 @@ jobs:
262262
persist-credentials: false
263263
- name: "Select toolchain"
264264
uses: dtolnay/rust-toolchain@5d458579430fc14a04a08a1e7d3694f545e91ce6 # stable
265-
- name: "Add architecture i386"
266-
run: sudo dpkg --add-architecture i386
265+
- name: "Add architecture i386 and install dependencies"
266+
run: |
267+
sudo dpkg --add-architecture i386
268+
sudo apt-get update -y
269+
sudo apt-get install -y \
270+
gcc-multilib \
271+
g++-multilib \
272+
libc6-dev-i386 \
273+
lib32stdc++-13-dev
267274
- name: "Install i686 gcc"
268275
run: sudo apt-get update -y && sudo apt-get install -y gcc-multilib
269276
- name: "Install target"

‎.gitignore‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,10 +15,7 @@ hashes/target
1515
# Test artifacts
1616
bitcoin/dep_test
1717
mutants.out*
18-
19-
# Fuzz artifacts
20-
hfuzz_target
21-
hfuzz_workspace
18+
fuzz/coverage
2219

2320
# Local tooling
2421
.maintainer-tools

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -314,7 +314,7 @@ the project to enable fine-grained unit testing is also an ongoing effort.
314314
Unit and integration tests are available for those interested, along with benchmarks. For project
315315
developers, especially new contributors looking for something to work on, we do:
316316

317-
- Fuzz testing with [`Honggfuzz`](https://github.com/rust-fuzz/honggfuzz-rs)
317+
- Fuzz testing with [`libfuzzer`](https://github.com/rust-fuzz/libfuzzer)
318318
- Mutation testing with [`cargo-mutants`](https://github.com/sourcefrog/cargo-mutants)
319319
- Code verification with [`Kani`](https://github.com/model-checking/kani)
320320

‎Cargo-minimal.lock‎

Lines changed: 6 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ dependencies = [
9898
"bitcoin",
9999
"bitcoin-consensus-encoding",
100100
"bitcoin-p2p-messages",
101-
"honggfuzz",
101+
"libfuzzer-sys",
102102
"serde",
103103
"serde_json",
104104
"standard_test",
@@ -264,43 +264,26 @@ version = "0.1.1"
264264
source = "registry+https://github.com/rust-lang/crates.io-index"
265265
checksum = "3011d1213f159867b13cfd6ac92d2cd5f1345762c63be3554e84092d85a50bbd"
266266

267-
[[package]]
268-
name = "honggfuzz"
269-
version = "0.5.58"
270-
source = "registry+https://github.com/rust-lang/crates.io-index"
271-
checksum = "6e8319f3cc8fe416e7aa1ab95dcc04fd49f35397a47d0b2f0f225f6dba346a07"
272-
dependencies = [
273-
"lazy_static",
274-
"memmap2",
275-
"rustc_version",
276-
"semver",
277-
]
278-
279267
[[package]]
280268
name = "itoa"
281269
version = "0.4.3"
282270
source = "registry+https://github.com/rust-lang/crates.io-index"
283271
checksum = "1306f3464951f30e30d12373d31c79fbd52d236e5e896fd92f96ec7babbbe60b"
284272

285-
[[package]]
286-
name = "lazy_static"
287-
version = "1.5.0"
288-
source = "registry+https://github.com/rust-lang/crates.io-index"
289-
checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
290-
291273
[[package]]
292274
name = "libc"
293275
version = "0.2.155"
294276
source = "registry+https://github.com/rust-lang/crates.io-index"
295277
checksum = "97b3888a4aecf77e811145cadf6eef5901f4782c53886191b2f693f24761847c"
296278

297279
[[package]]
298-
name = "memmap2"
299-
version = "0.9.0"
280+
name = "libfuzzer-sys"
281+
version = "0.4.0"
300282
source = "registry+https://github.com/rust-lang/crates.io-index"
301-
checksum = "deaba38d7abf1d4cca21cc89e932e542ba2b9258664d2a9ef0e61512039c9375"
283+
checksum = "86c975d637bc2a2f99440932b731491fc34c7f785d239e38af3addd3c2fd0e46"
302284
dependencies = [
303-
"libc",
285+
"arbitrary",
286+
"cc",
304287
]
305288

306289
[[package]]
@@ -358,15 +341,6 @@ dependencies = [
358341
"zerocopy",
359342
]
360343

361-
[[package]]
362-
name = "rustc_version"
363-
version = "0.4.0"
364-
source = "registry+https://github.com/rust-lang/crates.io-index"
365-
checksum = "bfa0f585226d2e68097d4f95d113b15b83a82e819ab25717ec0590d9584ef366"
366-
dependencies = [
367-
"semver",
368-
]
369-
370344
[[package]]
371345
name = "ryu"
372346
version = "1.0.0"
@@ -393,12 +367,6 @@ dependencies = [
393367
"cc",
394368
]
395369

396-
[[package]]
397-
name = "semver"
398-
version = "1.0.0"
399-
source = "registry+https://github.com/rust-lang/crates.io-index"
400-
checksum = "76b5842e81eb9bbea19276a9dbbda22ac042532f390a67ab08b895617978abf3"
401-
402370
[[package]]
403371
name = "serde"
404372
version = "1.0.195"

‎Cargo-recent.lock‎

Lines changed: 19 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ dependencies = [
9797
"bitcoin",
9898
"bitcoin-consensus-encoding",
9999
"bitcoin-p2p-messages",
100-
"honggfuzz",
100+
"libfuzzer-sys",
101101
"serde",
102102
"serde_json",
103103
"standard_test",
@@ -211,6 +211,8 @@ version = "1.1.22"
211211
source = "registry+https://github.com/rust-lang/crates.io-index"
212212
checksum = "9540e661f81799159abee814118cc139a2004b3a3aa3ea37724a1b66530b90e0"
213213
dependencies = [
214+
"jobserver",
215+
"libc",
214216
"shlex",
215217
]
216218

@@ -260,29 +262,20 @@ version = "0.1.1"
260262
source = "registry+https://github.com/rust-lang/crates.io-index"
261263
checksum = "3011d1213f159867b13cfd6ac92d2cd5f1345762c63be3554e84092d85a50bbd"
262264

263-
[[package]]
264-
name = "honggfuzz"
265-
version = "0.5.58"
266-
source = "registry+https://github.com/rust-lang/crates.io-index"
267-
checksum = "6e8319f3cc8fe416e7aa1ab95dcc04fd49f35397a47d0b2f0f225f6dba346a07"
268-
dependencies = [
269-
"lazy_static",
270-
"memmap2",
271-
"rustc_version",
272-
"semver",
273-
]
274-
275265
[[package]]
276266
name = "itoa"
277267
version = "1.0.11"
278268
source = "registry+https://github.com/rust-lang/crates.io-index"
279269
checksum = "49f1f14873335454500d59611f1cf4a4b0f786f9ac11f4312a78e4cf2566695b"
280270

281271
[[package]]
282-
name = "lazy_static"
283-
version = "1.5.0"
272+
name = "jobserver"
273+
version = "0.1.32"
284274
source = "registry+https://github.com/rust-lang/crates.io-index"
285-
checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
275+
checksum = "48d1dbcbbeb6a7fec7e059840aa538bd62aaccf972c7346c4d9d2059312853d0"
276+
dependencies = [
277+
"libc",
278+
]
286279

287280
[[package]]
288281
name = "libc"
@@ -291,19 +284,20 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
291284
checksum = "561d97a539a36e26a9a5fad1ea11a3039a67714694aaa379433e580854bc3dc5"
292285

293286
[[package]]
294-
name = "memchr"
295-
version = "2.7.4"
287+
name = "libfuzzer-sys"
288+
version = "0.4.10"
296289
source = "registry+https://github.com/rust-lang/crates.io-index"
297-
checksum = "78ca9ab1a0babb1e7d5695e3530886289c18cf2f87ec19a575a0abdce112e3a3"
290+
checksum = "5037190e1f70cbeef565bd267599242926f724d3b8a9f510fd7e0b540cfa4404"
291+
dependencies = [
292+
"arbitrary",
293+
"cc",
294+
]
298295

299296
[[package]]
300-
name = "memmap2"
301-
version = "0.9.5"
297+
name = "memchr"
298+
version = "2.7.4"
302299
source = "registry+https://github.com/rust-lang/crates.io-index"
303-
checksum = "fd3f7eed9d3848f8b98834af67102b720745c4ec028fcd0aa0239277e7de374f"
304-
dependencies = [
305-
"libc",
306-
]
300+
checksum = "78ca9ab1a0babb1e7d5695e3530886289c18cf2f87ec19a575a0abdce112e3a3"
307301

308302
[[package]]
309303
name = "ppv-lite86"
@@ -367,15 +361,6 @@ dependencies = [
367361
"getrandom",
368362
]
369363

370-
[[package]]
371-
name = "rustc_version"
372-
version = "0.4.1"
373-
source = "registry+https://github.com/rust-lang/crates.io-index"
374-
checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92"
375-
dependencies = [
376-
"semver",
377-
]
378-
379364
[[package]]
380365
name = "ryu"
381366
version = "1.0.18"
@@ -402,12 +387,6 @@ dependencies = [
402387
"cc",
403388
]
404389

405-
[[package]]
406-
name = "semver"
407-
version = "1.0.23"
408-
source = "registry+https://github.com/rust-lang/crates.io-index"
409-
checksum = "61697e0a1c7e512e84a621326239844a24d8207b4669b41bc18b32ea5cbf988b"
410-
411390
[[package]]
412391
name = "serde"
413392
version = "1.0.210"

‎Cross.toml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
[target.s390x-unknown-linux-gnu]
2+
dockerfile = ".github/docker/Dockerfile.cross"
3+
4+
[target.aarch64-unknown-linux-gnu]
5+
dockerfile = ".github/docker/Dockerfile.cross"

‎contrib/whitelist_deps.sh‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,5 @@
77
# - hashes because of secp v0.30
88
# - io because of hashes
99
# - hex because 1.0 only has decoding
10-
DUPLICATE_DEPS=("bitcoin_hashes" "hex-conservative" "bitcoin-io")
10+
# - libc v0.2.159 appears twice (runtime vs build-dependency) but is the same version, so Cargo deduplicates correctly
11+
DUPLICATE_DEPS=("bitcoin_hashes" "hex-conservative" "bitcoin-io" "libc")

0 commit comments

Comments
 (0)