chore(Node): Node does not keep a pointer to a GenesisDoc after initialization#4250
Merged
alesforz merged 10 commits intoalesforz/genesis-in-memoryfrom Oct 14, 2024
Merged
Conversation
added 5 commits
October 4, 2024 17:22
This reverts commit a99292d.
…ct that it returns as a singleton. The purpose of these changes is to set the `Node`'s pointer to the genesis to `nil` after initialization, allowing the garbage collector to deallocate the genesis struct from memory. In future work, we plan to remove this pointer entirely. Currently, different parts of the code call the `ConfigureRPC()` method to create an `Environment` object for their own purposes. However, this is wasteful because each `Environment` instance stores a copy of the genesis file: either as a single pointer or as a collection of chunks. Therefore, creating multiple, independent `Environment` objects results in multiple copies of the genesis being loaded into memory. For large genesis files, this can lead to large memory usage. By treating `Environment` as a singleton within `ConfigureRPC()`, we prevent multiple copies of the genesis from being loaded into memory simultaneously.
The `Node` does not need to access the genesis after starting up. If the genesis is small (<=16MB) then the `Environment` object serving the RPC APIs will retaing a pointer to the genesis at runtime. If the genesis is big (>16MB) then the `Environment` object will split the genesis into multiple chunks and keep them in memory.
5 tasks
This reverts commit 7613068. The method `GenesisDoc` is unexported, therefore some users might be using it. Deleting it might be a breaking change for them.
jmalicevic
approved these changes
Oct 14, 2024
Collaborator
jmalicevic
left a comment
There was a problem hiding this comment.
Needs a changelog ,entry, great work otherwise !
…error)`. Because `Node` does not store the genesis in memory anymore, this method now must retrieve the genesis from disk and unmarshal it into a `*types.GenesisDoc`. Note that this is a breaking, but necessary, change to ensure the correct behavior of `GenesisDoc()`.
alesforz
pushed a commit
that referenced
this pull request
Oct 14, 2024
alesforz
pushed a commit
that referenced
this pull request
Oct 16, 2024
…alization (#4250) Closes #4248. ### Changes The `Node` type will: - create a singleton instance of `Environment` when setting up the RPC APIs. - set the pointer to the `GenesisDoc` to `nil` once startup is complete so that the GC can collect the in-memory `GenesisDoc` object. - store a `GenesisTime` field. --- #### PR checklist - [x] Tests written/updated - [ ] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [x] Updated relevant documentation (`docs/` or `spec/`) and code comments
alesforz
pushed a commit
that referenced
this pull request
Oct 16, 2024
github-merge-queue bot
pushed a commit
that referenced
this pull request
Oct 25, 2024
…e operation (#4247) Closes #1290. This PR tracks the work we are doing for #1290. For details, please refer to #1290 discussion. ### ToDo - [x] Environment stores either a pointer to a GenesisDoc or the genesis' chunks, but not both. Merged to this branch with #4235. - [x] [Node](https://github.com/cometbft/cometbft/blob/87a8c8948bf9926aff37f344b4120163f12cb4ad/node/node.go#L50) does not keep a pointer to a GenesisDoc after initialization. Merged to this branch with #4250. --- #### PR checklist - [x] Tests written/updated - [x] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [x] Updated relevant documentation (`docs/` or `spec/`) and code comments --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
Eric-Warehime
pushed a commit
to Eric-Warehime/cometbft
that referenced
this pull request
Oct 28, 2024
…e operation (cometbft#4247) Closes cometbft#1290. This PR tracks the work we are doing for cometbft#1290. For details, please refer to cometbft#1290 discussion. ### ToDo - [x] Environment stores either a pointer to a GenesisDoc or the genesis' chunks, but not both. Merged to this branch with cometbft#4235. - [x] [Node](https://github.com/cometbft/cometbft/blob/87a8c8948bf9926aff37f344b4120163f12cb4ad/node/node.go#L50) does not keep a pointer to a GenesisDoc after initialization. Merged to this branch with cometbft#4250. --- #### PR checklist - [x] Tests written/updated - [x] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [x] Updated relevant documentation (`docs/` or `spec/`) and code comments --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4248.
Changes
The
Nodetype will:Environmentwhen setting up the RPC APIs.GenesisDoctonilonce startup is complete so that the GC can collect the in-memoryGenesisDocobject.GenesisTimefield.PR checklist
.changelog(we use unclog to manage our changelog)docs/orspec/) and code comments