-
Notifications
You must be signed in to change notification settings - Fork 4.1k
pkg/util/log: configuration for log redaction "sensitivity levels" #87038
Description
Is your feature request related to a problem? Please describe.
As our compliance requirements become more nuanced, we find ourselves heading in a direction where we want to be able to fine-tune how we redact logs.
For example, we have seen multiple PRs now that either attempt to redact conditionally, or explicitly mark certain types of data as safe by labeling them as operational:
- telemetry,sql: remove redaction from operational sql data #76676
- cli: support
COCKROACH_REDACTION_POLICY_MANAGEDenv var #86475
As we continue moving in this direction, if we're not mindful of how we approach these different redaction "sensitivity levels", we will find ourselves in a state of fragmented & disjoint implementations for these special cases. When there are only a couple cases to consider, sometimes the simple approach is appropriate. However, we are getting to a place where a more holistic approach is required to avoid future cruft.
Describe the solution you'd like
We should provide a way to configure redaction "sensitivity levels" within the database that can be easily used throughout the codebase. For example, SafeFormat implementations can have different outputs depending on the sensitivity level, where the logging clients can simply interpolate objects into logging format strings without explicitly marking them with things like SafeOperational (#76676) or SafeManaged (#86475).
We can do things like wrap redact.StringBuilder with our own log.StringBuilder that gains awareness of the redaction sensitivity level to cover cases such as calls to RedactableBytes.
Furthermore, as we aim to move towards a future where we can have multiple active logging configurations per-process (to support multiple tenants, per-process), this can provide a way to enable different redaction policies for each tenant.
Jira issue: CRDB-19115
Epic CRDB-39822