fix(util/http): default POST Content-Type to application/octet-stream#829
Merged
oskarszoon merged 1 commit intoMay 8, 2026
Merged
Conversation
DoHTTPRequest hardcoded Content-Type: application/json for any POST body. The only internal caller that sends a body is subtreevalidation.getMissingTransactionsBatch, which streams packed 32-byte tx hashes (binary). When that POST traverses a ModSecurity WAF in front of asset, the JSON body parser fails on binary data and Rule 200002 rejects the request with HTTP 400. Each rejection drops the source peer's reputation; with the WAF in place across the mainnet ingress the failure path eventually pushes peers below the selection threshold and starves catchup. Switch to application/octet-stream — matches every body shape this helper actually carries today. JSON callers do not exist; if one is added later it should set the header explicitly.
Contributor
|
🤖 Claude Code Review Status: Complete Current Review: No issues found. The change is correct and well-justified:
|
|
Contributor
Benchmark Comparison ReportBaseline: Current: Summary
All benchmark results (sec/op)
Threshold: >10% with p < 0.05 | Generated: 2026-05-07 18:23 UTC |
freemans13
approved these changes
May 7, 2026
icellan
approved these changes
May 7, 2026
icellan
approved these changes
May 7, 2026
| if len(requestBody) > 0 && requestBody[0] != nil { | ||
| req.Body = io.NopCloser(bytes.NewReader(requestBody[0])) | ||
| req.Method = http.MethodPost | ||
| req.Header.Set("Content-Type", "application/json") |
ordishs
added a commit
to ordishs/teranode
that referenced
this pull request
May 19, 2026
…efs #4571) Add TestCheckBlockSubtrees_OversizedBody to cover the third peer-fetch call site at check_block_subtrees.go:218, mirroring the existing fetchSubtreeFromPeer and getSubtreeTxHashes coverage so all three peer- fetch paths fail closed on oversized responses. Align TestDoHTTPRequestBounded_POST content-type assertion with the application/octet-stream default introduced upstream by bsv-blockchain#829, surfaced after rebasing onto main.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
util/http.go:DoHTTPRequesthardcodedContent-Type: application/jsonfor every POST request body. The only internal caller that sends a body via this helper issubtreevalidation.getMissingTransactionsBatch, which streams packed 32-byte tx hashes — binary data, never JSON.When that POST traverses a ModSecurity WAF (deployed in front of the mainnet asset ingress), the JSON request body parser is selected based on the header, fails on binary content, and Rule
200002(Failed to parse request body) denies the request with HTTP 400 + an HTML error page.The downstream effect is severe: each rejected
getMissingTransactionsBatchcall counts as a catchup failure against the source peer. Reputation math (successRate * 0.6 + baseScore * 0.4 - recentFailurePenalty) drives reputation toward 5.0 over time, eventually starving sync entirely. Observed in production onmainnet-eu-1peers (mainnet.gorillanode.io,mainnet2.gorillanode.io,bsva-ovh-teranode-eu-3).ModSecurity log excerpt:
Fix
Switch the default Content-Type from
application/jsontoapplication/octet-stream. This matches every body shape the helper actually carries today. There are no JSON-body callers — if one is added later it should set the header explicitly rather than relying on the helper default.Changes
util/http.go: change hardcodedContent-Typetoapplication/octet-stream, document the WAF interaction in a comment so future readers don't revert it.util/http_test.go: update the four POST test assertions that checked for the old header.Test plan
go test ./util/ -run 'TestDoHTTPRequest' -count=1— 29 passedgo vet ./util/...— cleangetMissingTransactionsBatchno longer 400s when traversing a ModSecurity-fronted ingressNotes
A complementary infra patch is being applied to the ModSecurity rule set on the affected ingress to add a path-scoped
requestBodyAccess=Offfor/api/v1/(subtree/{hash}/txs|tx|txs). That covers older peers still sending the wrong header until they upgrade past this PR.