Skip to content

Prevent OOM with bounded thread pool and memory cleanup#250

Merged
pda merged 2 commits intomainfrom
reduce-memory-usage
Feb 10, 2026
Merged

Prevent OOM with bounded thread pool and memory cleanup#250
pda merged 2 commits intomainfrom
reduce-memory-usage

Conversation

@joshuay03
Copy link
Contributor

@joshuay03 joshuay03 commented Feb 8, 2026

Context:

Addresses OOM issues by limiting concurrent uploads to 10 threads using concurrent-ruby (previously unbounded) and ensuring traces are cleaned from memory after upload. The unbounded thread creation was the likely culprit for memory exhaustion.

Also fixes a flaky test that was comparing exact gzipped bytes, which failed due to gzip timestamps. Now verifies decompressed content instead.

@joshuay03 joshuay03 self-assigned this Feb 8, 2026
@joshuay03 joshuay03 requested a review from a team as a code owner February 8, 2026 22:59
@joshuay03 joshuay03 force-pushed the reduce-memory-usage branch from 1ba7331 to eaebf52 Compare February 8, 2026 23:05
spec.add_dependency "concurrent-ruby"

spec.add_development_dependency "activesupport", ">= 4.2"
spec.add_development_dependency "ostruct"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is needed for Ruby >= 4.0. Ideally we'd make it conditional, but the main issue is that we check-in Gemfile.lock in this repo, which isn't ideal when supporting multiple Ruby versions, so leaving like this for now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we remove Gemfile.lock from git do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@pda pda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic, thanks @joshuay03 🙏🏼

I've tested this branch locally in our main test suite, seems good to me.

@pda pda merged commit 9233402 into main Feb 10, 2026
1 check passed
@pda pda deleted the reduce-memory-usage branch February 10, 2026 04:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants