perf(loki.secretfilter): Change secretfilter implementation to use Gitleaks by kleimkuhler · Pull Request #5503 · grafana/alloy

kleimkuhler · 2026-02-11T16:32:24Z

This PR refactors the loki.secretfilter component to use the Gitleaks library instead of a custom implementation.

Changes

Replaced custom regex-based secret detection with Gitleaks' detect.Detector
Arguments:
- Kept:
  - forward_to
  - origin_label
  - gitleaks_config (path to custom Gitleaks TOML)
  - redact_with (template with $SECRET_NAME / $SECRET_HASH)
- Added:
  - redact_percent (1–100, default 80; Gitleaks-style partial redaction)
- Removed:
  - partial_mask (replaced with redact_percent)
  - allowlist (controlled with custom gitleaks config)
  - enable_entropy (always enabled)
  - include_generic (controlled with custom gitleaks config)
  - types (controlled with custom gitleaks config)
Redaction: If redact_with is set, it is used as the placeholder; otherwise redaction is percentage-based via redact_percent (default 80%; 100% = full "REDACTED")
Metrics: keeping secrets_redacted_total, secrets_redacted_by_rule_total, secrets_redacted_by_origin (when origin_label is set), processing_duration_seconds. Removed secrets_allowlisted_total and secrets_skipped_entropy_by_rule_total as they are not efficient to determine with new library.
Docs: Reference updated for the new arguments and metrics

BREAKING-CHANGE: Some config options are removed entirely:

partial_mask (replaced with redact_percent)
allowlist (now controlled with custom gitleaks config)
enable_entropy
include_generic (now controlled with custom gitleaks config)
types (now controlled with custom gitleaks config)

BEGIN_COMMIT_OVERRIDE
feat(loki.secretfilter)!: Change secretfilter implementation to use Gitleaks (#5503)

BREAKING-CHANGE: Some config options are removed entirely:

partial_mask (replaced with redact_percent)
allowlist (now controlled with custom gitleaks config)
enable_entropy
include_generic (now controlled with custom gitleaks config)
types (now controlled with custom gitleaks config)

END_COMMIT_OVERRIDE

CLAassistant · 2026-02-11T16:32:32Z

All committers have signed the CLA.

github-actions · 2026-02-11T22:24:06Z

🔍 Dependency Review

No existing dependency versions were upgraded in this PR. The changes to go.mod files only add new (mostly indirect) dependencies and make one existing dependency (github.com/BurntSushi/toml v1.5.0) indirect in the root module without changing its version. Per the review rules, net‑new dependencies are not assessed for upgrade impact.

Therefore, there are no upgrade-related code changes required for existing dependencies.

🗒️ Notes

Net-new direct dependencies added in the root module (not assessed per rules):
- github.com/spf13/viper v1.19.0
- github.com/zricethezav/gitleaks/v8 v8.30.0
Numerous net-new indirect dependencies were added across modules (collector/, extension/alloyengine/, and the root module) to support new functionality (for example around secret detection/redaction and archive handling). Examples include:
- Detection/regex/WASM helpers: github.com/zricethezav/gitleaks/v8, github.com/gitleaks/go-gitdiff, github.com/wasilibs/go-re2, github.com/tetratelabs/wazero, github.com/wasilibs/wazero-helpers
- Config and utilities: github.com/spf13/viper, github.com/pelletier/go-toml/v2, github.com/spf13/afero
- Compression/archives: github.com/mholt/archives, github.com/therootcompany/xz, github.com/ulikunitz/xz, github.com/dsnet/compress, github.com/nwaples/rardecode/v2, github.com/sorairolake/lzip-go, github.com/STARRY-S/zip
- Misc: github.com/h2non/filetype, github.com/BobuSumisu/aho-corasick, github.com/sourcegraph/conc, github.com/fatih/semgroup, github.com/lucasb-eyer/go-colorful, github.com/muesli/termenv, github.com/xo/terminfo
One dependency changed from direct to indirect in the root module without a version change:
- github.com/BurntSushi/toml v1.5.0 → now marked indirect
- This does not require code changes.

If you intended to review upgrade impacts, please include diffs that change versions (old → new) for existing dependencies.

kalleep · 2026-02-12T08:23:37Z

(Breaking) Configuration options have been temporarily removed and will be added back in follow-up work

This component experimental so it's fine to make breaking changes. But if you plan to add back a all of a sub-set of the config I feel like there must be a better solution. If we merge this and the follow up work is not finished until next release we would make a release with breaking changes and then potentially add them back in a later release. That don't feel great for people using it.

As I said it's fine to make breaking changes but try to only do that if you have too. One option while not great is to keep the arguments for now and don't do anything. At least that won't break existing config even though the component internally have breaking changes

kelnage

Lots of comments - keeping the ability to specify a customer Gitleaks config is the key change, and replacing the Redact function with one that supports more of the secretfilter's functionality would also be a good idea

internal/component/loki/secretfilter/secretfilter.go

Better to reimplement again all features to maintain feature parity than risk a broken release

…ilter-gitleaks-implementation' into kleimkuhler/secretfilter-gitleaks-implementation

…tion

github-actions · 2026-02-13T04:10:06Z

💻 Deploy preview deleted (perf(loki.secretfilter): Change secretfilter implementation to use Gitleaks).

kelnage

Much better! A couple of minor points, I also noticed there was also a large reduction in the size of the unit tests, but I also noted the total code coverage increased, so I don't believe this is an issue.

docs/sources/reference/components/loki/loki.secretfilter.md

kelnage · 2026-02-13T10:55:08Z

internal/component/loki/secretfilter/gitleaks.toml

Note for other reviewers: this file is no longer required as it will take the default configuration (if used) directly from the Gitleaks package - so updating the github.com/zricethezav/gitleaks/v8 dependency will automatically update the default config to the latest version.

internal/component/loki/secretfilter/secretfilter_test.go

…tion

kelnage

LGTM!

kelnage

LGTM!

kalleep

LGTM but we do introduce breaking changes and we need to include that in changelog.

Read this to see how we can extend changelog entry beyond pr title

docs/sources/reference/components/loki/loki.secretfilter.md

Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

…ter-gitleaks-implementation

clayton-cornell

Doc changes look OK to me

…tleaks (#5503) This PR refactors the `loki.secretfilter` component to use the [Gitleaks](https://github.com/gitleaks/gitleaks) library instead of a custom implementation. ### Changes - Replaced custom regex-based secret detection with Gitleaks' `detect.Detector` - **Arguments:** - Kept: - `forward_to` - `origin_label` - `gitleaks_config` (path to custom Gitleaks TOML) - `redact_with` (template with `$SECRET_NAME` / `$SECRET_HASH`) - Added: - `redact_percent` (1–100, default 80; Gitleaks-style partial redaction) - Removed: - `partial_mask` (replaced with `redact_percent`) - `allowlist` (controlled with custom gitleaks config) - `enable_entropy` (always enabled) - `include_generic` (controlled with custom gitleaks config) - `types` (controlled with custom gitleaks config) - **Redaction:** If `redact_with` is set, it is used as the placeholder; otherwise redaction is percentage-based via `redact_percent` (default 80%; 100% = full `"REDACTED"`) - **Metrics:** keeping `secrets_redacted_total`, `secrets_redacted_by_rule_total`, `secrets_redacted_by_origin` (when `origin_label` is set), `processing_duration_seconds`. Removed `secrets_allowlisted_total` and `secrets_skipped_entropy_by_rule_total` as they are not efficient to determine with new library. - **Docs:** Reference updated for the new arguments and metrics BREAKING-CHANGE: Some config options are removed entirely: - `partial_mask` (replaced with `redact_percent`) - `allowlist` (now controlled with custom gitleaks config) - `enable_entropy` - `include_generic` (now controlled with custom gitleaks config) - `types` (now controlled with custom gitleaks config)

kleimkuhler changed the title ~~Change secretfilter implementation to use Gitleaks~~ perf(loki.secretfilter): Change secretfilter implementation to use Gitleaks Feb 11, 2026

kleimkuhler added 4 commits February 11, 2026 13:14

Change secretfilter implementation to use gitleaks

6e3bda0

-mod=mod build changes

a1ad714

Fix generate-module-dependencies

e64faae

make generate-otel-collector-distro

9915b07

kleimkuhler force-pushed the kleimkuhler/secretfilter-gitleaks-implementation branch from a7b3f0d to 9915b07 Compare February 11, 2026 20:14

Use DetectString

dd91714

mikefat previously approved these changes Feb 11, 2026

View reviewed changes

kleimkuhler marked this pull request as ready for review February 11, 2026 22:22

kleimkuhler requested a review from a team as a code owner February 11, 2026 22:22

kelnage self-requested a review February 12, 2026 09:45

kelnage requested changes Feb 12, 2026

View reviewed changes

kleimkuhler and others added 4 commits February 12, 2026 20:31

Restore component.redactLine

d25a794

Add gitleaks custom config

4172e1c

Update secretfilter docs with gitleaks implementation

5d1479d

Merge remote-tracking branch 'refs/remotes/origin/kleimkuhler/secretf…

39bb873

…ilter-gitleaks-implementation' into kleimkuhler/secretfilter-gitleaks-implementation

mikefat requested a review from clayton-cornell as a code owner February 13, 2026 04:07

Merge branch 'main' into kleimkuhler/secretfilter-gitleaks-implementa…

dfa98ef

…tion

mikefat added 3 commits February 12, 2026 22:32

Satisfy gofmt

9328408

rename config_path to gitleaks config

646a730

Satisfy gofmt

8152441

kelnage requested changes Feb 13, 2026

View reviewed changes

mikefat added 2 commits February 13, 2026 10:01

Update docs, address PR comments

5625eeb

Merge branch 'main' into kleimkuhler/secretfilter-gitleaks-implementa…

742be21

…tion

kelnage approved these changes Feb 13, 2026

View reviewed changes

kalleep reviewed Feb 16, 2026

View reviewed changes

clayton-cornell reviewed Feb 17, 2026

View reviewed changes

mikefat and others added 2 commits February 17, 2026 09:36

Apply suggestions from code review

3a4b3ab

Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Merge remote-tracking branch 'origin/main' into kleimkuhler/secretfil…

bd4adc9

…ter-gitleaks-implementation

clayton-cornell added the type/docs Docs Squad label across all Grafana Labs repos label Feb 18, 2026

clayton-cornell approved these changes Feb 18, 2026

View reviewed changes

kelnage requested a review from kalleep February 18, 2026 16:21

kleimkuhler merged commit e4907ab into main Feb 18, 2026
59 of 60 checks passed

kleimkuhler deleted the kleimkuhler/secretfilter-gitleaks-implementation branch February 18, 2026 16:29

grafana-alloybot bot mentioned this pull request Feb 18, 2026

Unreleased Changes #5170

Draft

jharvey10 mentioned this pull request Feb 18, 2026

fix: Add 'okok' line to wowfile.txt [backport] jharvey10/test-repo#171

Merged

jharvey10-test-app bot mentioned this pull request Feb 18, 2026

chore(release/v1.26): Release 1.26.2 jharvey10/test-repo#172

Merged

grafana-alloybot bot mentioned this pull request Feb 26, 2026

chore(release/v1.14): Release 1.14.0 #5662

Draft

Conversation

kleimkuhler commented Feb 11, 2026 • edited by jharvey10 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

CLAassistant commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Dependency Review

🗒️ Notes

Uh oh!

kalleep commented Feb 12, 2026

Uh oh!

kelnage left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kelnage left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kelnage Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kelnage left a comment

Choose a reason for hiding this comment

Uh oh!

kelnage left a comment

Choose a reason for hiding this comment

Uh oh!

kalleep left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

clayton-cornell left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

kleimkuhler commented Feb 11, 2026 •

edited by jharvey10

Loading

CLAassistant commented Feb 11, 2026 •

edited

Loading

github-actions bot commented Feb 11, 2026 •

edited

Loading

github-actions bot commented Feb 13, 2026 •

edited

Loading