Skip to content

storageccl: Allow online manual encryption key rotation of all files #74804

@itsbilal

Description

@itsbilal

Currently, with encryption-at-rest enabled using --enterprise-encryption, the only way we rotate store / data keys is when a new sstable or WAL is written. This form of lazy encryption works well in the general case, but for cases where a customer wants to enable encryption on all files quickly without waiting for them to be rewritten, there's no direct way to do that. Similarly, #65798 describes the other direction - how it's difficult to remove encryption from a store because stale files will stay encrypted.

We need to add a manual process that lets us re-encrypt all existing sstables with new data keys (or decrypt, if --enterprise-encryption=newkey=plaintext). Ideally we would be able to do this on an online db without taking each node down, but adding a command that requires taking the node offline (like cockroach debug compact) would be a fair start.

Jira issue: CRDB-12274

Epic CRDB-16419

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-storageRelating to our storage engine (Pebble) on-disk storage.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)T-storageStorage Team

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions