Skip to content

storage: optimize new SST iterators #83051

@erikgrinaker

Description

@erikgrinaker

The new SST iterators in #82799, e.g. NewPebbleSSTMemIterator(), are much slower than the existing NewMemSSTIterator(), around 80% or so. Some of this is due to CRDB-side overhead, in particular the use of pebbleIterator and VerifyingMVCCIterator, but the majority appears to be in Pebble via the new NewExternalIter() SST iterator. We need to optimize these.

This can be seen with BenchmarkSSTIterator, by changing NewMemSSTIterator() to NewPebbleMemSSTIterator() in runSSTIterator() on top of #82799.

Many of the optimizations that address #82559 and #83049 would likely apply here as well.

Jira issue: CRDB-16822

Epic CRDB-2624

Metadata

Metadata

Assignees

Labels

A-kv-replicationRelating to Raft, consensus, and coordination.A-storageRelating to our storage engine (Pebble) on-disk storage.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)C-performancePerf of queries or internals. Solution not expected to change functional behavior.T-storageStorage Team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions