storage: expose additional engine metrics#46558
storage: expose additional engine metrics#46558craig[bot] merged 1 commit intocockroachdb:masterfrom
Conversation
petermattis
left a comment
There was a problem hiding this comment.
This looks very much like what I was expecting. I have a few small comments below.
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @itsbilal and @jbowens)
pkg/kv/kvserver/metrics.go, line 354 at r1 (raw file):
} metaRdbCompactionBytesIngested = metric.Metadata{ Name: "rocksdb.compaction.bytes-ingested",
Ingested and compaction are separate concepts. Ingested bytes are not compacted. Compacted bytes are not ingested. I think this should be named rocksdb.ingested-bytes instead. The Help text will also need adjustment. It might be worthwhile chatting with myself or @itsbilal about the difference between ingestion and compaction.
pkg/storage/engine.go, line 486 at r1 (raw file):
MemtableTotalSize int64 Flushes int64 FlushBytes int64
Nit: Let's consistently use past tense. This would become FlushedBytes.
pkg/storage/engine.go, line 488 at r1 (raw file):
FlushBytes int64 Compactions int64 CompactionBytesIngested int64 // (Pebble only)
Similar to other comment, this should be IngestedBytes.
pkg/storage/engine.go, line 490 at r1 (raw file):
CompactionBytesIngested int64 // (Pebble only) CompactionBytesRead int64 CompactionBytesWritten int64
Nit: per above: CompactedBytesRead and CompactedBytesWritten.
pkg/ts/catalog/chart_catalog.go, line 1838 at r1 (raw file):
}, { Title: "Flushed Bytes",
How do these sectionDescriptions show up in the admin UI? Compaction and flushing are related, so I could see a combined Flush and Compaction section instead of having them separate.
df2dd56 to
489fb14
Compare
jbowens
left a comment
There was a problem hiding this comment.
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @itsbilal and @petermattis)
pkg/kv/kvserver/metrics.go, line 354 at r1 (raw file):
Previously, petermattis (Peter Mattis) wrote…
Ingested and compaction are separate concepts. Ingested bytes are not compacted. Compacted bytes are not ingested. I think this should be named
rocksdb.ingested-bytesinstead. TheHelptext will also need adjustment. It might be worthwhile chatting with myself or @itsbilal about the difference between ingestion and compaction.
Done.
pkg/storage/engine.go, line 488 at r1 (raw file):
Previously, petermattis (Peter Mattis) wrote…
Similar to other comment, this should be
IngestedBytes.
Done.
pkg/ts/catalog/chart_catalog.go, line 1838 at r1 (raw file):
Previously, petermattis (Peter Mattis) wrote…
How do these
sectionDescriptionsshow up in the admin UI? Compaction and flushing are related, so I could see a combinedFlush and Compactionsection instead of having them separate.
As far as I can tell, these sectionDescriptions are unused. They're surfaced by the /_admin/v1/chartcatalog endpoint, but I wasn't unable to find any consumers of that endpoint. The structure of the graphs under the metrics tab looks like it is specified within the Admin UI's JSX (dashboards/storage.tsx).
There is still a unit test requiring that every metric appear in the chart catalog, which is why I added these here. I'm not sure if that's just a vestige from an older iteration of the Admin UI? The custom chart endpoint also appears to get its list of metrics through a separate /_admin/v1/metricmetadata endpoint, which pulls its data right from the registry, not the catalog.
petermattis
left a comment
There was a problem hiding this comment.
modulo one final nit to resolve
Reviewable status:
complete! 1 of 0 LGTMs obtained (waiting on @itsbilal and @jbowens)
pkg/kv/kvserver/metrics.go, line 360 at r2 (raw file):
} metaRdbCompactedBytesRead = metric.Metadata{ Name: "rocksdb.compaction.bytes-read",
Nit: should this be rocksdb.compacted-bytes-read? That would be more consistent with the variable names and the rocksdb.{flushed,ingested}-bytes metrics. I don't believe the dots signify anything particularly important other than grouping. And not all of the existing compaction metrics have the rocksdb.compaction prefix (e.g. rocksdb.estimated-pending-compaction).
pkg/ts/catalog/chart_catalog.go, line 1838 at r1 (raw file):
Previously, jbowens (Jackson Owens) wrote…
As far as I can tell, these sectionDescriptions are unused. They're surfaced by the /_admin/v1/chartcatalog endpoint, but I wasn't unable to find any consumers of that endpoint. The structure of the graphs under the metrics tab looks like it is specified within the Admin UI's JSX (dashboards/storage.tsx).
There is still a unit test requiring that every metric appear in the chart catalog, which is why I added these here. I'm not sure if that's just a vestige from an older iteration of the Admin UI? The custom chart endpoint also appears to get its list of metrics through a separate /_admin/v1/metricmetadata endpoint, which pulls its data right from the registry, not the catalog.
Ack. Thanks for taking a look.
itsbilal
left a comment
There was a problem hiding this comment.
I imagine we'd have to ditch the rocksdb. prefix for all metrics at some point too, but that can wait for some point in 20.2.
Reviewed 1 of 8 files at r1, 7 of 7 files at r2.
Reviewable status:complete! 2 of 0 LGTMs obtained (waiting on @jbowens)
Release justification: low-risk update to new functionality Fixes cockroachdb#43965 Release note: none
|
bors r+ |
Build failed (retrying...) |
Build succeeded |
Release justification: low-risk update to new functionality
Fixes #43965
Release note: none