Skip to content

Add redact struct tag for automatic field redaction during Unpack#224

Draft
Copilot wants to merge 17 commits intomainfrom
copilot/add-redact-tag-option
Draft

Add redact struct tag for automatic field redaction during Unpack#224
Copilot wants to merge 17 commits intomainfrom
copilot/add-redact-tag-option

Conversation

Copy link
Copy Markdown

Copilot AI commented Feb 18, 2026 •
edited
Loading

Adds a redact struct tag option that automatically redacts sensitive fields to "[REDACTED]" during Unpack() operations. A new ShowRedacted option bypasses redaction when needed.

Implementation

Struct tag parsing (util.go):

  • Added redact to supported tag options
  • Parsed alongside existing options like inline, validate, etc.

Metadata tracking (merge.go):

  • Extended Meta struct with Redacted bool field
  • normalizeValue() marks fields with redact tag in metadata
  • Config stores original values; redaction deferred to unpack time

Redaction at unpack (reify.go):

  • doReifyPrimitive(): Redacts primitive string/[]byte/[]rune fields
  • reifySliceMerge(): Handles []byte and []rune redaction
  • reifyValue(): Redacts when unpacking to map[string]interface{}
  • Checks metadata.Redacted flag and opts.showRedacted option

ShowRedacted option (opts.go):

  • New option for Unpack() to show original unredacted values
  • Default behavior (no option): fields are redacted
  • With option: original values preserved during unpack

Supported Types

Redaction applies to fields of type:

  • string
  • []byte
  • []rune
  • Custom types based on the above (e.g., type Password string)

Other types ignore the redact tag.

Usage

type Config struct {
    Username string `config:"username"`
    Password string `config:"password,redact"`
    APIKey   []byte `config:"api_key,redact"`
}

cfg, _ := ucfg.NewFrom(Config{
    Username: "admin",
    Password: "secret123",
    APIKey:   []byte("key"),
})

// Default: redacted
var c1 Config
cfg.Unpack(&c1)
// c1.Password = "[REDACTED]"
// c1.APIKey = []byte("[REDACTED]")

// With ShowRedacted: originals
var c2 Config
cfg.Unpack(&c2, ucfg.ShowRedacted)
// c2.Password = "secret123"
// c2.APIKey = []byte("key")

Tests

Added 11 tests covering:

  • Default redaction behavior
  • ShowRedacted option
  • Nested structs, arrays, inline structs
  • Type restrictions (only string/[]byte/[]rune)
  • Custom types (type aliases)
  • Mixed redacted/non-redacted fields

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 3 commits February 18, 2026 21:58
@michel-laterman
Add redact struct tag option and Redact() method
0b9c4d7 
Co-authored-by: michel-laterman <82832767+michel-laterman@users.noreply.github.com>
@michel-laterman
Fix trailing whitespace issues
e3ac05c 
Co-authored-by: michel-laterman <82832767+michel-laterman@users.noreply.github.com>
@michel-laterman
Add additional tests for inline structs and idempotency
c41c2e8 
Co-authored-by: michel-laterman <82832767+michel-laterman@users.noreply.github.com>
Copilot AI changed the title [WIP] Add new struct tag option named redact Add redact struct tag and Config.Redact() method Feb 18, 2026
@ebeahan ebeahan added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Feb 19, 2026
@michel-laterman michel-laterman marked this pull request as ready for review February 19, 2026 16:59
@michel-laterman michel-laterman requested a review from a team as a code owner February 19, 2026 16:59
@michel-laterman michel-laterman requested review from blakerouse and ycombinator and removed request for michel-laterman February 19, 2026 16:59
@michel-laterman michel-laterman mentioned this pull request Feb 19, 2026
Merged
2 tasks
ycombinator
ycombinator requested changes Feb 20, 2026
View reviewed changes
Copy link
Copy Markdown

@ycombinator ycombinator left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation is easy to understand and tests are clear.

Redaction is for security so I'm wondering if makes sense for it to be the default behavior. Concretely, should we introduce a new Option to prevent redaction when parsing a config using the NewFrom and MustNewFrom constructors, rather than relying on the user to explicitly remember to call Redact() after constructing the Config object? I imagine we'd want to provide a way to access the unredacted fields as well.

@cmacknz
Copy link
Copy Markdown
Member

cmacknz commented Feb 20, 2026

Agree, the need to explicitly call cfg.Redact makes this harder to use safely because it can be forgotten.

For comparison, the collector version of this has an explicit type whose methods prevent the value from becoming visible, see here. This is much safer if we could do something like this instead, especially if we already know we must annotate every type anyway.

Additionally, if I look at files included in diagnostics, they are marshalled to YAML and don't seem to pass through ucfg for example here.

So I think I would like some clear examples of where we would insert the cfg.Redact calls to make sure that API would immediately help us. I like the concept but there does seem like a better version of this.

Copilot AI and others added 3 commits February 23, 2026 19:43
@michel-laterman
Implement option-based redact approach with ShowRedacted option
64cb9fb 
Co-authored-by: michel-laterman <82832767+michel-laterman@users.noreply.github.com>
@michel-laterman
Address code review feedback - fix metadata logic and improve test na…
b440e52 
…ming

Co-authored-by: michel-laterman <82832767+michel-laterman@users.noreply.github.com>
@michel-laterman
Add documentation for redact tag and ShowRedacted option
554578d 
Co-authored-by: michel-laterman <82832767+michel-laterman@users.noreply.github.com>
Copilot AI changed the title Add redact struct tag and Config.Redact() method Refactor redact to option-based approach with secure-by-default behavior Feb 23, 2026
@michel-laterman
Remove Redact method and associated tests
a87cfbb 
Co-authored-by: michel-laterman <82832767+michel-laterman@users.noreply.github.com>
blakerouse
blakerouse requested changes Feb 23, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if this is working as I would expect. I would not expect that when I perform a New that the content of Config to contain only the redacted values. I would have expected it to contain the actual values.

If I then use the config with Unpack into a structure, then I would expect that it perform the redaction unless the ShowRedacted is included in the option of the unpack.

ycombinator
ycombinator reviewed Feb 23, 2026
View reviewed changes
ycombinator
ycombinator reviewed Feb 23, 2026
View reviewed changes
Copy link
Copy Markdown

@ycombinator ycombinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a minor refactoring suggestion but LGTM overall.

@michel-laterman
Move redaction from New/Merge to Unpack and add []rune support
213a0a2 
Co-authored-by: michel-laterman <82832767+michel-laterman@users.noreply.github.com>
Copilot AI changed the title Add redact tag to string and []byte types, redact by default Move redaction to Unpack and add []rune support Feb 23, 2026
@michel-laterman
Rename tests and fix comments to reflect Unpack-based redaction
32cfb51 
Co-authored-by: michel-laterman <82832767+michel-laterman@users.noreply.github.com>
Copilot AI changed the title Move redaction to Unpack and add []rune support Add redact struct tag with Unpack-time redaction for sensitive fields Feb 24, 2026
@michel-laterman
Fix ShowRedacted comment in opts.go to reflect Unpack-time redaction
ab7754d 
Co-authored-by: michel-laterman <82832767+michel-laterman@users.noreply.github.com>
Copilot AI changed the title Add redact struct tag with Unpack-time redaction for sensitive fields Add redact struct tag for automatic redaction during Unpack Feb 24, 2026
blakerouse
blakerouse requested changes Feb 24, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks better, very close. I have one more ask that I believe will required code changes.

@michel-laterman
Add unit test for custom types with redaction
71bb76d 
Co-authored-by: michel-laterman <82832767+michel-laterman@users.noreply.github.com>
Copilot AI changed the title Add redact struct tag for automatic redaction during Unpack Add redact struct tag for automatic field redaction during Unpack Feb 24, 2026
blakerouse
blakerouse approved these changes Feb 24, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all the fixes and cleanups. This looks good!

@cmacknz
Copy link
Copy Markdown
Member

cmacknz commented Feb 24, 2026

Additionally, if I look at files included in diagnostics, they are marshalled to YAML and don't seem to pass through ucfg for example here.

Am I missing something, or is redacting during Unpack not going to solve the diagnostics redaction problem here? We get those configs long after they passed through Unpack and because the running agent/beat system needs the real values that original Unpack will have happened with the ucfg.ShowRedacted?

I can see how this works, and I can see why adding struct tags to control redaction seems useful, but I missing where in our code we would use this and it would solve a problem for us. We would need a place where we call Unpack and then immediately try to print the complete configuration? Where does that happen today?

@cmacknz
Copy link
Copy Markdown
Member

cmacknz commented Feb 24, 2026

I think my main problem with this right now is every real use of Unpack is going to set ucfg.ShowRedacted because it needs the actual values and not just the redacted string, thus making the redacted tag useless unless you re-unpack before printing or marshalling which seems very unlikely to be a thing that happens reliably.

@michel-laterman
Copy link
Copy Markdown

michel-laterman commented Feb 24, 2026

@cmacknz that's a good point; i'll mark this as a draft for the time being, we can discuss the path forward/if this is even needed in the weekly meeting

@michel-laterman michel-laterman marked this pull request as draft February 24, 2026 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ycombinator ycombinator ycombinator approved these changes

@blakerouse blakerouse blakerouse approved these changes

@michel-laterman michel-laterman Awaiting requested review from michel-laterman

Assignees

@michel-laterman michel-laterman

Copilot code review Copilot

Labels

Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@cmacknz @michel-laterman @ycombinator @blakerouse @ebeahan