Skip to content

storage: remove end-key comparison in ComputeStatsForRange #41899

@petermattis

Description

@petermattis

On every iteration inside of the main loop in ComputeStatsForRange, a comparison is done against the end key. This is usually unnecessary as the iterator that has been passed to ComputeStatsForRange usually has had an UpperBound set. The performance impact of removing this comparison is large:

name                                      old time/op    new time/op    delta
MVCCComputeStats_Pebble/valueSize=8-16       159ms ± 4%     115ms ± 8%  -27.72%  (p=0.000 n=10+10)
MVCCComputeStats_Pebble/valueSize=32-16      110ms ± 7%      78ms ± 3%  -29.73%  (p=0.000 n=10+10)
MVCCComputeStats_Pebble/valueSize=256-16    35.2ms ±11%    26.5ms ±15%  -24.79%  (p=0.000 n=10+10)

name                                      old speed      new speed      delta
MVCCComputeStats_Pebble/valueSize=8-16     424MB/s ± 4%   586MB/s ± 8%  +38.47%  (p=0.000 n=10+10)
MVCCComputeStats_Pebble/valueSize=32-16    609MB/s ± 8%   866MB/s ± 3%  +42.18%  (p=0.000 n=10+10)
MVCCComputeStats_Pebble/valueSize=256-16  1.91GB/s ±11%  2.55GB/s ±13%  +33.30%  (p=0.000 n=10+10)

The challenge to removing it is that some uses of ComputeStatsForRange use iterators which don't have an upper-bound. For example, engine/bulk.AddSSTable.

Jira issue: CRDB-5409

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-performancePerf of queries or internals. Solution not expected to change functional behavior.T-storageStorage Team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions