feat: Optimize UniqMap to reduce unnecessary slice preallocation#710
Merged
samber merged 3 commits intosamber:masterfrom Oct 12, 2025
Merged
feat: Optimize UniqMap to reduce unnecessary slice preallocation#710samber merged 3 commits intosamber:masterfrom
samber merged 3 commits intosamber:masterfrom
Conversation
Contributor
Author
func TestUniqMapOld_LenCap(t *testing.T) {
t.Parallel()
is := assert.New(t)
type User struct {
Name string
age int
}
users := make([]User, 100_000)
for i := 0; i < 100_000; i++ {
users[i] = User{Name: "Alex", age: 25}
}
users[50_000] = User{Name: "Alice", age: 25}
result := UniqMapOld(users, func(item User, index int) string {
return item.Name
})
slices.Sort(result)
is.Equal([]string{"Alex", "Alice"}, result)
is.Equal(2, len(result))
is.Equal(100_000, cap(result))
}
func TestUniqMap_LenCap(t *testing.T) {
t.Parallel()
is := assert.New(t)
type User struct {
Name string
age int
}
users := make([]User, 100_000)
for i := 0; i < 100_000; i++ {
users[i] = User{Name: "Alex", age: 25}
}
users[50_000] = User{Name: "Alice", age: 25}
result := UniqMap(users, func(item User, index int) string {
return item.Name
})
slices.Sort(result)
is.Equal([]string{"Alex", "Alice"}, result)
is.Equal(2, len(result))
is.Equal(2, cap(result))
} |
9a2cf3a to
890e997
Compare
ivolkoff
commented
Oct 11, 2025
ivolkoff
commented
Oct 11, 2025
Owner
|
thanks ! |
samber
requested changes
Oct 12, 2025
Owner
samber
left a comment
There was a problem hiding this comment.
Go < 1.21 does not support slices.Sort
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #710 +/- ##
==========================================
+ Coverage 94.50% 94.53% +0.02%
==========================================
Files 18 18
Lines 3439 3437 -2
==========================================
- Hits 3250 3249 -1
+ Misses 175 174 -1
Partials 14 14
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
The current implementation of
UniqMapOldpreallocates the result slice with the same capacity as the input collection, even though the number of unique elements is usually much smaller.This leads to excessive memory allocation and higher memory usage.
The proposed change adjusts the allocation strategy to use a slice sized according to the number of unique elements, significantly reducing memory footprint without changing functionality.
⚙️ What Changed
Before (UniqMapOld):
After (UniqMap):
📊 Benchmark Comparison
BenchmarkUniqMapOldBenchmarkUniqMapImprovement:
5.1MB → 3.5MB)✅ Test Results
TestUniqMapOld_LenCapcap(result) == 100_000TestUniqMap_LenCapcap(result) == 2💡 Motivation
Using a smaller result slice capacity directly proportional to the number of unique elements significantly reduces memory consumption — especially for large input slices with high repetition rates — without affecting correctness or performance negatively.
🧠 Summary