db: add range key table stats#1616
Conversation
|
This change is |
jbowens
left a comment
There was a problem hiding this comment.
Reviewed 1 of 1 files at r1, 10 of 11 files at r2, all commit messages.
Reviewable status: 10 of 11 files reviewed, 4 unresolved discussions (waiting on @itsbilal and @nicktrav)
-- commits, line 5 at r1:
s/range key/range deletion/
-- commits, line 7 at r1:
Just a heads up, for producing delete-only compactions we'll need to examine both range deletions and range keys in tandem, which might throw a wrench in factoring range keys into their own method.
table_stats.go, line 294 at r1 (raw file):
} // loadTablePointKeyStats calculates the range deletion statistics for the given
s/loadTablePointKeyStats/sloadTableRangeDelStats/
testdata/table_stats, line 184 at r2 (raw file):
Previously, nicktrav (Nick Travers) wrote…
Curiously, these crept in due to the use of the
testkeys.Comparer. I need to dig into why exactly, but it seems benign.
I bet it's just the comparer name which gets serialized into the sstable properties. The range-deletion-bytes-estimate includes the full size of sstables that are wholly contained by a rangedel, so that does include the properties block.
To improve readability, factor out the logic to populate the point and range deletion table stats into their own separate methods. This also allows for adding a separate method for range keys in the future, if the logic becomes complex enough.
Range key-related statistics are not currently calculated and stored in a table's `FileMetadata`, and are required for compaction picking. Add a `NumRangeKeys` field to `manifest.TableStats` which is populated by the total number of range keys present in a table (the sum of range key sets, unsets and deletes).
f342710 to
a2e377b
Compare
nicktrav
left a comment
There was a problem hiding this comment.
TFTR!
Reviewable status: 0 of 11 files reviewed, 3 unresolved discussions (waiting on @itsbilal and @jbowens)
Previously, jbowens (Jackson Owens) wrote…
s/range key/range deletion/
Done.
Previously, jbowens (Jackson Owens) wrote…
Just a heads up, for producing delete-only compactions we'll need to examine both range deletions and range keys in tandem, which might throw a wrench in factoring range keys into their own method.
Ah - good to know. Will see how this plays out.
table_stats.go, line 294 at r1 (raw file):
Previously, jbowens (Jackson Owens) wrote…
s/loadTablePointKeyStats/sloadTableRangeDelStats/
Done.
testdata/table_stats, line 184 at r2 (raw file):
Previously, jbowens (Jackson Owens) wrote…
I bet it's just the comparer name which gets serialized into the sstable properties. The range-deletion-bytes-estimate includes the full size of sstables that are wholly contained by a rangedel, so that does include the properties block.
Ah nice.
func TestLen(t *testing.T) {
fmt.Println(len(testkeys.Comparer.Name)) // 24
fmt.Println(len(base.DefaultComparer.Name)) // 26
}
Two commits in this series - the first is a code shuffle (no logic changes); the second has the additions.
db: factor out some table stats helpers
To improve readability, factor out the logic to populate the point and
range key table stats into their own separate methods. This also allows
for adding a separate method for range keys in the future, if the logic
becomes complex enough.
db: add range key count to table stats
Range key-related statistics are not currently calculated and stored in
a table's
FileMetadata, and are required for compaction picking.Add a
NumRangeKeysfield tomanifest.TableStatswhich is populatedby the total number of range keys present in a table (the sum of range
key sets, unsets and deletes).