Skip to content

db: add block bytes to IteratorStats #1342

@sumeerbhola

Description

@sumeerbhola

The current IteratorStats count seeks and steps. There is a desire to also include the total bytes encountered, since rows can be big. We could expose two things:

  • [A] Cumulative key+value size for each position the Iterator was at (this could be calculated by the caller, but probably better in the lower layer).
  • [B] Cumulative bytes in blocks that were loaded using sstable.Reader.readBlock.

This desire is mentioned

  • as a bullet in storage: track iteration stats per engine.Iterator cockroach#59069 (comment)
  • implied in an internal conversation with @andreimatei "if anyone ends up doing more work in the area, I've got a feature request for ya - I want data on the size of the skipped garbage".
    • Pebble is not concerned with what is "skipped" by the MVCC layer, but by exposing A and B, we can know that B-A was wasted. Though such waste may not have been costly if it hit the block cache.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions