-
Notifications
You must be signed in to change notification settings - Fork 780
Closed
Labels
P:storage-optimizationPriority: Give operators greater control over storage and storage optimizationPriority: Give operators greater control over storage and storage optimizationP:tech-debtPriority: Technical debt that needs to be paid off to enable us to move faster, reliablyPriority: Technical debt that needs to be paid off to enable us to move faster, reliablybacklogA prioritized task in the team's backlogA prioritized task in the team's backlogrpc
Description
As part of #1037, we need to optimize how the genesis file is chunked.
we need to implement logic to:
- Determine the number of chunks (based on the chunk size and file size). This logic already exists, might need to adapt for optimal chunk size.
- Store the chunks in the file system (TBD in the
tmpordatafolder inside home) - Return chunks loaded from the file system (not from storage) for the
genesis_chunkedendpoint. - [OPTIONAL] Validate chunks (re-create the file from chunks) and hash it to ensure it matches hash in storage or hash of genesis in the file system (
configfolder). Maybe this could be validated when the node starts to ensure nothing has been tampered with.
DOD
- Do not store chunked genesis files in memory.
- Store the chunks on storage or file system and fetch them on demand.
- Rethink the default size of the genesis chunk and use an appropriate value.
This completes the work started in Phase II of #1037, thus finalizing the implementation of the chunking strategy discussed here.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
P:storage-optimizationPriority: Give operators greater control over storage and storage optimizationPriority: Give operators greater control over storage and storage optimizationP:tech-debtPriority: Technical debt that needs to be paid off to enable us to move faster, reliablyPriority: Technical debt that needs to be paid off to enable us to move faster, reliablybacklogA prioritized task in the team's backlogA prioritized task in the team's backlogrpc