-
Notifications
You must be signed in to change notification settings - Fork 4.1k
storage: remove end-key comparison in ComputeStatsForRange #41899
Copy link
Copy link
Labels
C-performancePerf of queries or internals. Solution not expected to change functional behavior.Perf of queries or internals. Solution not expected to change functional behavior.T-storageStorage TeamStorage Team
Description
On every iteration inside of the main loop in ComputeStatsForRange, a comparison is done against the end key. This is usually unnecessary as the iterator that has been passed to ComputeStatsForRange usually has had an UpperBound set. The performance impact of removing this comparison is large:
name old time/op new time/op delta
MVCCComputeStats_Pebble/valueSize=8-16 159ms ± 4% 115ms ± 8% -27.72% (p=0.000 n=10+10)
MVCCComputeStats_Pebble/valueSize=32-16 110ms ± 7% 78ms ± 3% -29.73% (p=0.000 n=10+10)
MVCCComputeStats_Pebble/valueSize=256-16 35.2ms ±11% 26.5ms ±15% -24.79% (p=0.000 n=10+10)
name old speed new speed delta
MVCCComputeStats_Pebble/valueSize=8-16 424MB/s ± 4% 586MB/s ± 8% +38.47% (p=0.000 n=10+10)
MVCCComputeStats_Pebble/valueSize=32-16 609MB/s ± 8% 866MB/s ± 3% +42.18% (p=0.000 n=10+10)
MVCCComputeStats_Pebble/valueSize=256-16 1.91GB/s ±11% 2.55GB/s ±13% +33.30% (p=0.000 n=10+10)
The challenge to removing it is that some uses of ComputeStatsForRange use iterators which don't have an upper-bound. For example, engine/bulk.AddSSTable.
Jira issue: CRDB-5409
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
C-performancePerf of queries or internals. Solution not expected to change functional behavior.Perf of queries or internals. Solution not expected to change functional behavior.T-storageStorage TeamStorage Team