Skip to content

fix: release response memory immediately to prevent OOM#2118

Merged
nimrod-teich merged 3 commits into
mainfrom
fix/oom-release-response-copies
Nov 30, 2025
Merged

fix: release response memory immediately to prevent OOM#2118
nimrod-teich merged 3 commits into
mainfrom
fix/oom-release-response-copies

Conversation

@nimrod-teich

@nimrod-teich nimrod-teich commented Nov 26, 2025

Copy link
Copy Markdown
Contributor

Description

Closes: #XXXX


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • read the contribution guide
  • included the correct type prefix in the PR title, you can find examples of the prefixes below:
  • confirmed ! in the type prefix if API or client breaking change
  • targeted the main branch
  • provided a link to the relevant issue or specification
  • reviewed "Files changed" and left comments if necessary
  • included the necessary unit and integration tests
  • updated the relevant documentation or specification, including comments for documenting Go code
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic, API design and naming, documentation is accurate, tests and test coverage

Note

Reduce peak memory by clearing large JSON-RPC Result fields ASAP and unmarshalling only error fields; minor test timing tweaks.

  • JSON-RPC memory optimizations
    • Parse error-only structures in CheckResponseError and CheckResponseErrorForJsonRpcBatch to avoid allocating large result payloads (protocol/chainlib/chainproxy/rpcInterfaceMessages/jsonRPCMessage.go).
    • After converting responses, immediately clear large Result fields to enable GC:
      • In ConvertJsonRPCMsg, set rpcMsg.Result = nil post-conversion.
      • In client batch handling (rpcclient/client.go: BatchCallContext), nil resp.Result after unmarshalling into targets.
      • In proxy paths (protocol/chainlib/jsonRPC.go):
        • Batch: convert elements then set batch[idx].Result = nil; after marshaling replies, clear replyMsgs[idx].Result.
        • Single call: after ID validation, set rpcMessage.Result = nil; after marshaling, clear replyMsg.Result.
  • Tests
    • Adjust smartrouter mock ticker durations to 10s to align with new processing timing (protocol/rpcsmartrouter/smartrouter_relay_state_machine_test.go).

Written by Cursor Bugbot for commit d3e3beb. This will update automatically on new commits. Configure here.

@pull-request-size pull-request-size Bot added size/M and removed size/S labels Nov 26, 2025
@nimrod-teich nimrod-teich changed the title fix: release batch response memory immediately to prevent OOM fix: release response memory immediately to prevent OOM Nov 26, 2025

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the final PR Bugbot will review for you during this billing cycle

Your free Bugbot reviews will reset on December 23

Details

You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

Comment thread protocol/chainlib/chainproxy/rpcclient/http.go Outdated
Comment thread protocol/chainlib/chainproxy/rpcclient/http.go Outdated
Comment thread protocol/chainlib/jsonRPC.go Outdated
Comment thread protocol/chainlib/jsonRPC.go Outdated
Comment thread protocol/chainlib/jsonRPC.go Outdated
@github-actions

github-actions Bot commented Nov 26, 2025

Copy link
Copy Markdown

Test Results

3 089 tests  ±0   3 088 ✅ ±0   31m 43s ⏱️ - 3m 28s
  126 suites ±0       1 💤 ±0 
    7 files   ±0       0 ❌ ±0 

Results for commit d3e3beb. ± Comparison against base commit 1087ad5.

♻️ This comment has been updated with latest results.

@nimrod-teich nimrod-teich force-pushed the fix/oom-release-response-copies branch from 6df5122 to b9e2ae5 Compare November 27, 2025 15:15
Release large JSON-RPC response memory immediately after consumption to prevent
out-of-memory issues with large payloads.

Changes:
- Clear resp.Result after unmarshaling in BatchCallContext
- Clear source rpcMessage.Result after conversion in ConvertJsonRPCMsg
- Clear batch[].Result immediately after conversion in sendBatchMessage
- Clear replyMsgs after successful marshaling
- Clear rpcMessage and replyMsg after all references complete in SendNodeMsg

All cleanup happens AFTER data consumption to ensure correctness while
reducing memory pressure from large JSON payloads across all chain types.
@nimrod-teich nimrod-teich force-pushed the fix/oom-release-response-copies branch from b9e2ae5 to 78b9b73 Compare November 27, 2025 15:19
Use specialized structs in CheckResponseError and CheckResponseErrorForJsonRpcBatch to avoid unmarshaling the Result field, preventing unnecessary memory allocation for large payloads during error checks.
NadavLevi
NadavLevi previously approved these changes Nov 30, 2025
@nimrod-teich nimrod-teich force-pushed the fix/oom-release-response-copies branch from 2a2ec90 to d3e3beb Compare November 30, 2025 13:07
@nimrod-teich nimrod-teich merged commit d440d2c into main Nov 30, 2025
32 checks passed
@nimrod-teich nimrod-teich deleted the fix/oom-release-response-copies branch November 30, 2025 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants