feat(alloc): new allocator crate#402
Merged
Merged
Conversation
09926b4 to
4ff260b
Compare
danielsn
reviewed
May 1, 2024
| /// new [LinearAllocator] to the previous one, creating a chain. This is where | ||
| /// its name comes from. | ||
| pub struct ChainAllocator<A: Allocator + Clone> { | ||
| top: UnsafeCell<ChainNodePtr<A>>, |
Contributor
Author
There was a problem hiding this comment.
If you think of the chain as a stack, this is the one on "top".
| /// is worth it. This is somewhat arbitrarily chosen at the moment. | ||
| const MIN_NODE_SIZE: usize = 4 * size_of::<Self>(); | ||
|
|
||
| /// Creates a new [ChainAllocator]. The `chunk_size_hint` is used as a |
Contributor
There was a problem hiding this comment.
Any advice on how to choose/tune this?
Contributor
Author
There was a problem hiding this comment.
I clarified a bit in the docs. Let me know if you want more advice here.
e5524f2 to
1196378
Compare
0f98317 to
622db33
Compare
This will be used with the new StringTable API for datadog-profiling. The Chain- and LinearAllocators support querying the number of used and reserved bytes. They are arena allocators, which means they do not deallocate individual items, and do batched deallocation when they drop.
622db33 to
b68fe99
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #402 +/- ##
==========================================
+ Coverage 65.49% 66.28% +0.78%
==========================================
Files 184 187 +3
Lines 22565 23118 +553
==========================================
+ Hits 14780 15324 +544
- Misses 7785 7794 +9
|
735c623 to
248bbe1
Compare
r1viollet
approved these changes
May 10, 2024
r1viollet
left a comment
Contributor
There was a problem hiding this comment.
LGTM
Thanks for providing allocators!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds a new
datadog-alloccrate. There are three allocators:LinearAllocator<A: Allocator>ChainAllocator<A: Allocator + Clone>VirtualAllocatorThe
ChainAllocatorandLinearAllocatorsupport querying the number of used and reserved bytes. They are arena allocators, which means they do not deallocate individual items, and do batched deallocation when they drop.Each of these allocators is further documented in the code.
Motivation
This will be used with the new StringTable API for datadog-profiling. Avoiding the system allocator has benefits. In particular, sometimes with the individual string allocations it would look like there was a memory leak but was just the system allocator holding onto memory we previously allocated without returning it to the system.
Being able to query used and reserved memory can be useful for stats. The PHP profiler intends to use these to conditionally reset its string cache.
Additional Notes
The allocator API is not stable, so this uses the allocator-api2 which exports the same API for stable usage.
Here is a string table diagram that shows how these allocators get used:
How to test the change?
Regular cargo tests.
For Reviewers
@DataDog/security-design-and-guidance.