-
Notifications
You must be signed in to change notification settings - Fork 50
Open
Labels
I4No visible changesNo visible changesS1Highly significantHighly significantU3RegularRegularenhancementImproving existing functionalityImproving existing functionalityneofs-storageStorage node application issuesStorage node application issuesperformanceMore of something per secondMore of something per second
Description
Is your feature request related to a problem? Please describe.
I'm always frustrated when I look at how we allocate a lot of memory to fetch data from storage. #2316, #2178, #1398. #2814 also reminds about it.
Describe the solution you'd like
#1932 has some valid input for these problems, although it's localized to CLI, whereas I'm more concerned about SN. We can store/fetch objects in a bit different manner, serializing header and payload separately right at the FSTree level. Then object get operation would return a header and an io.ReaderCloser pointing to the file (if it's not too small). This can then be streamed to the outside as needed (but likely API changes are required as well).
Describe alternatives you've considered
Keep pulling MBs of data from the storage is no fun.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
I4No visible changesNo visible changesS1Highly significantHighly significantU3RegularRegularenhancementImproving existing functionalityImproving existing functionalityneofs-storageStorage node application issuesStorage node application issuesperformanceMore of something per secondMore of something per second