build(setup): pin + verify SHA256 for all third_party downloads (#3665)#3692
Conversation
Every scripts/setup/* download built/bundled into release artifacts without any integrity check — a compromised CDN, hijacked release asset, or TLS-MITM could substitute a malicious artifact and nothing would catch it. Pin the expected hash next to each URL/version and verify after download, before extract/build, failing hard on mismatch. - Shared verifiers: _verify_sha256.sh (verify_sha256, sha256sum/shasum) and _verify_sha256.ps1 (Confirm-Sha256, Get-FileHash) so the scripts don't each hand-roll it. - SHA256-pinned (verified by downloading each artifact): onnxruntime 1.18.1, fftw 3.3.5 dll64, hidapi 0.14.0, opus 1.5.2 (one hash covers both mirrors — confirmed identical), qtkeychain 0.16.0 (ps1 tarball), and all libdeepfilter dfnr-libs platform tarballs (linux x86_64/aarch64, darwin arm64/x86_64, windows x86_64). - Git-clone cases pin the immutable commit instead of a file hash: setup-qtkeychain.sh verifies the 0.16.0 tag resolves to its pinned commit; setup-deepfilter's source-build fallback was already pinned to DFNR_COMMIT. - deepfilter download hard-fails on hash mismatch (tampering) rather than silently falling back to a source build. Verified locally: verify_sha256 passes a correct artifact and rejects a deliberately-wrong hash; bash scripts pass `bash -n`. PowerShell paths validate in Windows CI. Note: the two GitHub auto-generated source archives (hidapi, qtkeychain.ps1) are pinned too; if GitHub ever regenerates those archives the hash must be re-pinned (version bumps already require that). Closes #3665. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Hi @ten9876 — thanks for tackling #3665! Pinning + verifying SHA256 on every setup download is exactly the kind of supply-chain hardening this repo needs, and the shared-verifier approach is clean. 🙏 The CI failure is in the contributor code, not CI infrastructure — and it's a small, expected one given the PR's own verification note. What failed: Why: The only change to verify_sha256 "/tmp/$TARBALL" "${DFNR_TARBALL_SHA256[$PLATFORM]:-}" || exit 1That same download already worked on macOS before this PR (it falls back to a source build only when This lines up with your own Verification section, which says the hashes were "proven locally against the real linux-x86_64 deepfilter tarball" — only To see the exact mismatch, open the failing step's log and look for the How to fix — re-pin the darwin hashes from the real release assets, the same way you did for linux-x86_64: # from the dfnr-libs release (use your DFNR_RELEASE_REPO, default ten9876/AetherSDR)
for P in darwin-arm64 darwin-x86_64; do
curl -fsSL -o "/tmp/libdeepfilter-$P.tar.gz" \
"https://github.com/ten9876/AetherSDR/releases/download/dfnr-libs/libdeepfilter-$P.tar.gz"
echo "$P: $(shasum -a 256 "/tmp/libdeepfilter-$P.tar.gz" | cut -d' ' -f1)"
doneThen update the One small robustness note while you're in there: when Everything else in the PR looks solid — the shared 🤖 aethersdr-agent · cost: $2.1970 · model: claude-opus-4-8 |
…bash 3.2)
check-macos failed: setup-deepfilter.sh used a bash-4 associative array
(declare -A), but macOS ships bash 3.2 which rejects it ("declare: -A: invalid
option"), aborting the script under set -e. Replace it with a dfnr_tarball_sha256()
case function — same per-platform pins, bash 3.2-safe. Verified no other
bash-4-only constructs remain in scripts/setup/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Reviewed the full diff plus the two git clone paths that aren't in the diff. This is solid supply-chain hardening — clean, well-scoped, and the design decisions are right.
Completeness — verified. I grepped scripts/setup/ for every download-bearing line and each one is now covered:
- The 8
Invoke-WebRequest/curlfile downloads each verify before extract/build. - The two
git clonecases pin the immutable commit, not just a mutable tag:setup-qtkeychain.shre-checksgit rev-parse HEADagainstQTKEYCHAIN_COMMIT, and the deepfilter source-build fallback was alreadygit checkout "$DFNR_COMMIT"(confirmed atsetup-deepfilter.sh:94/setup-deepfilter.ps1). No unverified download remains.
Things I liked:
- Hard-failing the deepfilter prebuilt on hash mismatch (tamper) while still letting a network failure fall back to source build is exactly the right distinction — and the success path
exit 0s before the fallback, so a verified prebuilt never accidentally rebuilds. - The
case-baseddfnr_tarball_sha256fails closed: an unmatched platform returns"", which can't match a real hash, so it aborts rather than skipping verification. Good defensive default. bash -n/ stock-bash-3.2 compatibility called out and respected (no associative arrays).
One minor, non-blocking note (your call): in the .ps1 scripts the verification sits inside if (-not (Test-Path $TarFile)), so a tarball already present from a prior run or a CI cache is reused without re-verification. The modeled threat (download-time MITM / compromised upstream) is fully covered since the file was verified when first fetched; a poisoned local cache is a different, lower-severity threat. Worth a one-line awareness comment at most — not something to block on.
One thing I couldn't check: I can't independently confirm the pinned hash values are correct (this environment blocks codeload.github.com / api.github.com / fftw.org). I'm trusting your stated methodology — that each hash came from actually downloading the artifact, and that the opus pin is byte-identical across both mirrors. The verification logic is sound; the values rest on that.
Nice work, @ten9876 — thanks for closing the gap on all of them in one pass rather than piecemeal.
🤖 aethersdr-agent · cost: $2.7982 · model: claude-opus-4-8
|
Adding issue #3590 as this should solve the binary vendoring audit / tracking issues. |
Summary
Closes #3665. Every
scripts/setup/*script downloaded a build dependency and built/bundled it into release artifacts without any integrity check — a compromised upstream/CDN, a hijacked release asset, or a TLS-MITM could substitute a malicious artifact and nothing would catch it. This pins the expected hash next to each URL/version and verifies after download, before extract/build, failing hard on mismatch.What changed
_verify_sha256.sh→verify_sha256 <file> <hex>(sha256sum, shasum fallback)_verify_sha256.ps1→Confirm-Sha256 -Path -Expected(Get-FileHash)libdeepfilterdfnr-libsplatform tarballs (linux x86_64/aarch64, darwin arm64/x86_64, windows x86_64).setup-qtkeychain.shverifies the0.16.0tag resolves to its pinned commit (aa6da344…);setup-deepfilter's source-build fallback was already pinned toDFNR_COMMIT.Verification
verify_sha256passes a correct artifact and rejects a deliberately-wrong hash (the issue's acceptance check) — proven locally against the real linux-x86_64 deepfilter tarball.bash -nclean on all bash scripts; every download-bearing script now has an adjacent verification (no unverified download remains). PowerShell paths validate in Windows CI.Note
The two GitHub auto-generated source archives (hidapi, qtkeychain.ps1) are pinned too; if GitHub ever regenerates those archives the hash must be re-pinned — but version bumps already require re-pinning, so they move together.
🤖 Generated with Claude Code