Skip to content

Capture index usage metrics in telemetry logging channel #72486

@kevin-v-ngo

Description

@kevin-v-ngo

This issue tracks capturing index usage metrics for all customer workloads. This will help us determine how many customers are suffering from unused indexes and measure our progress in helping developers design and tune their indexing strategy. We should take at least 8 hour periodic snapshots of this information

We should capture the envelope including the following event payload:

CREATE TABLE telemetry_logs_index_usage_statistics_raw (
database_name NOT NULL STRING,
database_id INT8 NOT NULL,
table_name NOT NULL STRING,
table_id INT8 NOT NULL,
ti.index_name INT8 NOT NULL,
index_id INT8 NOT NULL,
index_type NOT NULL STRING,
total_reads INT8 NOT NULL,
is_unique BOOL NOT NULL,
is_inverted BOOL NOT NULL,
last_read TIMESTAMPTZ NULL
)

crdb_internal tables that contain this information:

  • crdb_internal.index_usage_statistics
  • crdb_internal.table_indexes
  • crdb_internal.databases

Sample user query:

SELECT
current_database() as database_name,
 ti.descriptor_name as table_name,
 ti.descriptor_id as table_id,
 ti.index_name,
 ti.index_id,
 ti.index_type,
 ti.is_unique,
 ti.is_inverted,
 total_reads,
 last_read
FROM crdb_internal.index_usage_statistics AS us
JOIN crdb_internal.table_indexes ti
ON us.index_id = ti.index_id
 AND us.table_id = ti.descriptor_id
ORDER BY total_reads ASC;

Epic: CRDB-10495

Jira issue: CRDB-11177

Metadata

Metadata

Assignees

Labels

A-sql-logging-and-telemetryIssues related to slow query log, SQL audit log, SQL internal logging telemetry, etc.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions