Skip to content

Perf: Reduce memory overhead in transitive dependency#78651

Closed
toddtreece wants to merge 2 commits intomainfrom
toddtreece/arrow-json
Closed

Perf: Reduce memory overhead in transitive dependency#78651
toddtreece wants to merge 2 commits intomainfrom
toddtreece/arrow-json

Conversation

@toddtreece
Copy link
Copy Markdown
Member

@toddtreece toddtreece commented Nov 24, 2023

What is this feature?

image

Reduces go_memstats_heap_inuse_bytes from ~65MB to ~45MB on a mostly idle (except profiling) Grafana instance, by switching to lazy initialization of global variables in go-json.

Related PR: goccy/go-json#490

Why do we need this feature?

go-json is a dependency of Apache Arrow v13. Grafana does not currently use Arrow's JSON encoding/decoding, but Arrow's import of go-json is adding additional memory overhead due to how some global variables are initialized.

This is pretty easy to spot when looking at Pyroscope flamegraphs:
image

Who is this feature for?

[Add information on what kind of user the feature is for.]

Which issue(s) does this PR fix?:

Fixes #

Special notes for your reviewer:

Please check that:

  • It works as expected from a user's perspective.
  • If this is a pre-GA feature, it is behind a feature toggle.
  • The docs are updated, and if this is a notable improvement, it's added to our What's New doc.

@grafana-delivery-bot grafana-delivery-bot Bot added this to the 10.3.x milestone Nov 24, 2023
@toddtreece toddtreece force-pushed the toddtreece/arrow-json branch from 19fce78 to 0cdd5ca Compare December 19, 2023 02:43
@toddtreece toddtreece added no-backport Skip backport of PR no-changelog Skip including change in changelog/release notes labels Dec 19, 2023
@grafana grafana deleted a comment from ephemeral-instances-bot Bot Dec 19, 2023
@grafana grafana deleted a comment from ephemeral-instances-bot Bot Dec 19, 2023
@toddtreece
Copy link
Copy Markdown
Member Author

/deploy-to-hg

@ephemeral-instances-bot
Copy link
Copy Markdown

  • Preparing your instance. A comment containing your instance's url will be added to this PR when the instance is ready.
  • Your instance will be ready in ~10 minutes.
  • Check the GitHub actions tab to follow the workflow progress
  • Slack channel: #proj-ephemeral-hg-instances
  • Building instance with toddtreece/arrow-json oss branch and main enterprise branch. How to choose a branch

@toddtreece toddtreece marked this pull request as ready for review December 19, 2023 03:07
@toddtreece toddtreece requested a review from a team as a code owner December 19, 2023 03:07
@toddtreece toddtreece requested review from a team, idafurjes, yangkb09 and zserge and removed request for a team December 19, 2023 03:07
@toddtreece toddtreece changed the title Perf: Reduce Apache Arrow memory overhead Perf: Reduce memory overhead in transitive dependency Dec 19, 2023
@ephemeral-instances-bot
Copy link
Copy Markdown

@github-actions
Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 2 weeks if no further activity occurs. Please feel free to give a status update or ping for review. Thank you for your contributions!

@github-actions github-actions Bot added the stale Issue with no recent activity label Feb 25, 2024
@github-actions
Copy link
Copy Markdown
Contributor

This pull request has been automatically closed because it has not had any further activity in the last 2 weeks. Thank you for your contributions!

@github-actions github-actions Bot closed this Mar 11, 2024
@yuri-tceretian yuri-tceretian modified the milestones: 10.3.x, 10.3.5 Mar 20, 2024
@joshhunt joshhunt deleted the toddtreece/arrow-json branch June 25, 2025 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/backend no-backport Skip backport of PR no-changelog Skip including change in changelog/release notes stale Issue with no recent activity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants