Skip to content

pkg/util/log: configuration for log redaction "sensitivity levels" #87038

@abarganier

Description

@abarganier

Is your feature request related to a problem? Please describe.
As our compliance requirements become more nuanced, we find ourselves heading in a direction where we want to be able to fine-tune how we redact logs.

For example, we have seen multiple PRs now that either attempt to redact conditionally, or explicitly mark certain types of data as safe by labeling them as operational:

  1. telemetry,sql: remove redaction from operational sql data #76676
  2. cli: support COCKROACH_REDACTION_POLICY_MANAGED env var #86475

As we continue moving in this direction, if we're not mindful of how we approach these different redaction "sensitivity levels", we will find ourselves in a state of fragmented & disjoint implementations for these special cases. When there are only a couple cases to consider, sometimes the simple approach is appropriate. However, we are getting to a place where a more holistic approach is required to avoid future cruft.

Describe the solution you'd like
We should provide a way to configure redaction "sensitivity levels" within the database that can be easily used throughout the codebase. For example, SafeFormat implementations can have different outputs depending on the sensitivity level, where the logging clients can simply interpolate objects into logging format strings without explicitly marking them with things like SafeOperational (#76676) or SafeManaged (#86475).

We can do things like wrap redact.StringBuilder with our own log.StringBuilder that gains awareness of the redaction sensitivity level to cover cases such as calls to RedactableBytes.

Furthermore, as we aim to move towards a future where we can have multiple active logging configurations per-process (to support multiple tenants, per-process), this can provide a way to enable different redaction policies for each tenant.

Jira issue: CRDB-19115

Epic CRDB-39822

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-observability-infC-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)T-supportability

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions