Skip to content

kv: NodeStatus keys not cleared on decommissioning #51304

@nvb

Description

@nvb

We don't currently clear entries from "status-node-" keyspace when a node is removed from the cluster. This can cause confusion when performing aggregation over those keys. For instance, when working with crdb_internal.kv_store_status and aggregating over metrics->>'ranges.underreplicated'.

I don't know the full history or intention of these keys and the metrics they contain, but we should likely either:

  1. clear the metrics portion of the keys associated with decommissioned nodes
  2. delete the keys associated with decommissioned nodes entirely

cc. @tbg since you might be one of the few people who still knows how this subsystem works and what its intentions are.

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-kv-decom-rolling-restartDecommission and Rolling RestartsA-kv-serverRelating to the KV-level RPC serverC-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)S-3-ux-surpriseIssue leaves users wondering whether CRDB is behaving properly. Likely to hurt reputation/adoption.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions