Skip to content

fix(redact): remove re.IGNORECASE from _ENV_ASSIGN_RE to prevent masking lowercase variables#4476

Closed
gnanam1990 wants to merge 1 commit into
NousResearch:mainfrom
gnanam1990:fix/redaction-ignorecase-4367
Closed

fix(redact): remove re.IGNORECASE from _ENV_ASSIGN_RE to prevent masking lowercase variables#4476
gnanam1990 wants to merge 1 commit into
NousResearch:mainfrom
gnanam1990:fix/redaction-ignorecase-4367

Conversation

@gnanam1990

@gnanam1990 gnanam1990 commented Apr 1, 2026

Copy link
Copy Markdown
Contributor

Root Cause

_ENV_ASSIGN_RE was compiled with re.IGNORECASE, making the uppercase-only pattern [A-Z_]*TOKEN[A-Z_]* also match lowercase names. This caused two distinct bugs:

Bugs Fixed

#4367 — Python variable assignments incorrectly redacted

before_tokens = response.usage.prompt_tokens  # was redacted to: before_tokens=***
api_key = config.get('api_key')               # was redacted to: api_key=***

#4451 — await keyword corrupted in TypeScript/TSX patch tool output

// Before (upstream bug):
const token = await getToken();  →  const token=*** getToken();
const secret = await fetchSecret();  →  const secret=*** fetchSecret();

// After (this fix):
const token = await getToken();  →  const token = await getToken();  ✓

The await keyword was being redacted because token (lowercase) matched the pattern with re.IGNORECASE, and await was the first non-whitespace word after =, so _mask_token("await") → *** (5 chars < 18).

Fix

Two changes to _ENV_ASSIGN_RE:

  1. Remove re.IGNORECASE — only ALL-UPPERCASE env var names should match (e.g. API_KEY, SECRET_TOKEN). Lowercase Python/JS/TS variable names are never redacted.
  2. Add (?:^|(?<=\s)) lookbehind — prevents the pattern from consuming leading whitespace, so export SECRET_TOKEN=value stays export SECRET_TOKEN=*** (not exportSECRET_TOKEN=***).

Tests

Added 5 new regression tests in TestEnvAssignments:

All 43 tests pass.

Fixes #4367
Fixes #4451

🤖 Generated with Claude Code

@gnanam1990 @claude
fix(redact): remove re.IGNORECASE from _ENV_ASSIGN_RE to prevent mask…
09b78c2 
…ing lowercase variables

The _ENV_ASSIGN_RE pattern was compiled with re.IGNORECASE, causing it to
match lowercase variable assignments like `token = await ...` and
`before_tokens = response.usage` as if they were secret environment variables.

This caused two reported bugs:
- NousResearch#4367: Python variable assignments (before_tokens, api_key, my_token)
  being incorrectly redacted in logs and tool output
- NousResearch#4451: TypeScript/JS `await` keyword corrupted to `***` in patch tool output
  because `const token = await getToken()` matched the pattern, replacing
  `await` with `***` and stripping the surrounding whitespace

Fix: remove re.IGNORECASE so only ALL-UPPERCASE env var names match.
Add (?:^|(?<=\s)) lookbehind to prevent the pattern from consuming
leading whitespace (e.g. `export SECRET=...` preserved correctly).

Adds regression tests covering both Python and TypeScript/JS cases.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
teknium1 added a commit that referenced this pull request Apr 5, 2026
@teknium1 @gnanam1990
test(redact): add regression tests for lowercase variable redaction (#…
585be63 
…4367)

Add 5 regression tests from PR #4476 (gnanam1990) to prevent re-introducing
the IGNORECASE bug that caused lowercase Python/TypeScript variable assignments
to be incorrectly redacted as secrets. The core fix landed in 6367e1c.

Tests cover:
- Lowercase Python variable with 'token' in name
- Lowercase Python variable with 'api_key' in name
- TypeScript 'await' not treated as secret value
- TypeScript 'secret' variable assignment
- 'export' prefix preserved for uppercase env vars

Co-authored-by: gnanam1990 <gnanam1990@users.noreply.github.com>
This was referenced Apr 5, 2026
Merged
Closed
@teknium1

teknium1 commented Apr 5, 2026

Copy link
Copy Markdown
Contributor

The core fix already landed on main in commit 6367e1c by @LucidPaths. Your regression tests were the most comprehensive of the three submissions — we salvaged all 5 tests into PR #5185 with your co-authorship preserved. Thanks for the thorough test coverage!

@teknium1 teknium1 closed this Apr 5, 2026
teknium1 added a commit that referenced this pull request Apr 5, 2026
@teknium1 @gnanam1990
test(redact): add regression tests for lowercase variable redaction (#…
5b003ca 
…4367) (#5185)

Add 5 regression tests from PR #4476 (gnanam1990) to prevent re-introducing
the IGNORECASE bug that caused lowercase Python/TypeScript variable assignments
to be incorrectly redacted as secrets. The core fix landed in 6367e1c.

Tests cover:
- Lowercase Python variable with 'token' in name
- Lowercase Python variable with 'api_key' in name
- TypeScript 'await' not treated as secret value
- TypeScript 'secret' variable assignment
- 'export' prefix preserved for uppercase env vars

Co-authored-by: gnanam1990 <gnanam1990@users.noreply.github.com>
Tommyeds pushed a commit to Tommyeds/hermes-agent that referenced this pull request Apr 12, 2026
@teknium1 @gnanam1990
test(redact): add regression tests for lowercase variable redaction (N…
9369a25 
…ousResearch#4367) (NousResearch#5185)

Add 5 regression tests from PR NousResearch#4476 (gnanam1990) to prevent re-introducing
the IGNORECASE bug that caused lowercase Python/TypeScript variable assignments
to be incorrectly redacted as secrets. The core fix landed in 6367e1c.

Tests cover:
- Lowercase Python variable with 'token' in name
- Lowercase Python variable with 'api_key' in name
- TypeScript 'await' not treated as secret value
- TypeScript 'secret' variable assignment
- 'export' prefix preserved for uppercase env vars

Co-authored-by: gnanam1990 <gnanam1990@users.noreply.github.com>
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
@teknium1 @gnanam1990
test(redact): add regression tests for lowercase variable redaction (N…
6d56606 
…ousResearch#4367) (NousResearch#5185)

Add 5 regression tests from PR NousResearch#4476 (gnanam1990) to prevent re-introducing
the IGNORECASE bug that caused lowercase Python/TypeScript variable assignments
to be incorrectly redacted as secrets. The core fix landed in 332c115.

Tests cover:
- Lowercase Python variable with 'token' in name
- Lowercase Python variable with 'api_key' in name
- TypeScript 'await' not treated as secret value
- TypeScript 'secret' variable assignment
- 'export' prefix preserved for uppercase env vars

Co-authored-by: gnanam1990 <gnanam1990@users.noreply.github.com>
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 28, 2026
@teknium1 @gnanam1990
test(redact): add regression tests for lowercase variable redaction (N…
77df740 
…ousResearch#4367)

Add 5 regression tests from PR NousResearch#4476 (gnanam1990) to prevent re-introducing
the IGNORECASE bug that caused lowercase Python/TypeScript variable assignments
to be incorrectly redacted as secrets. The core fix landed in 332c115.

Tests cover:
- Lowercase Python variable with 'token' in name
- Lowercase Python variable with 'api_key' in name
- TypeScript 'await' not treated as secret value
- TypeScript 'secret' variable assignment
- 'export' prefix preserved for uppercase env vars

Co-authored-by: gnanam1990 <gnanam1990@users.noreply.github.com>
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
@teknium1 @gnanam1990
test(redact): add regression tests for lowercase variable redaction (N…
4ea0995 
…ousResearch#4367) (NousResearch#5185)

Add 5 regression tests from PR NousResearch#4476 (gnanam1990) to prevent re-introducing
the IGNORECASE bug that caused lowercase Python/TypeScript variable assignments
to be incorrectly redacted as secrets. The core fix landed in 6367e1c.

Tests cover:
- Lowercase Python variable with 'token' in name
- Lowercase Python variable with 'api_key' in name
- TypeScript 'await' not treated as secret value
- TypeScript 'secret' variable assignment
- 'export' prefix preserved for uppercase env vars

Co-authored-by: gnanam1990 <gnanam1990@users.noreply.github.com>
olympus-terminal pushed a commit to olympus-terminal/hermes-agent that referenced this pull request May 16, 2026
@teknium1 @gnanam1990
test(redact): add regression tests for lowercase variable redaction (N…
2d2e1f4 
…ousResearch#4367) (NousResearch#5185)

Add 5 regression tests from PR NousResearch#4476 (gnanam1990) to prevent re-introducing
the IGNORECASE bug that caused lowercase Python/TypeScript variable assignments
to be incorrectly redacted as secrets. The core fix landed in 80522ed.

Tests cover:
- Lowercase Python variable with 'token' in name
- Lowercase Python variable with 'api_key' in name
- TypeScript 'await' not treated as secret value
- TypeScript 'secret' variable assignment
- 'export' prefix preserved for uppercase env vars

Co-authored-by: gnanam1990 <gnanam1990@users.noreply.github.com>
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
@teknium1 @gnanam1990
test(redact): add regression tests for lowercase variable redaction (N…
2170db4 
…ousResearch#4367) (NousResearch#5185)

Add 5 regression tests from PR NousResearch#4476 (gnanam1990) to prevent re-introducing
the IGNORECASE bug that caused lowercase Python/TypeScript variable assignments
to be incorrectly redacted as secrets. The core fix landed in 6367e1c.

Tests cover:
- Lowercase Python variable with 'token' in name
- Lowercase Python variable with 'api_key' in name
- TypeScript 'await' not treated as secret value
- TypeScript 'secret' variable assignment
- 'export' prefix preserved for uppercase env vars

Co-authored-by: gnanam1990 <gnanam1990@users.noreply.github.com>
Egavasyug pushed a commit to Egavasyug/hermes-agent that referenced this pull request Jun 10, 2026
@teknium1 @gnanam1990
test(redact): add regression tests for lowercase variable redaction (N…
0c0b55a 
…ousResearch#4367) (NousResearch#5185)

Add 5 regression tests from PR NousResearch#4476 (gnanam1990) to prevent re-introducing
the IGNORECASE bug that caused lowercase Python/TypeScript variable assignments
to be incorrectly redacted as secrets. The core fix landed in 454d48b.

Tests cover:
- Lowercase Python variable with 'token' in name
- Lowercase Python variable with 'api_key' in name
- TypeScript 'await' not treated as secret value
- TypeScript 'secret' variable assignment
- 'export' prefix preserved for uppercase env vars

Co-authored-by: gnanam1990 <gnanam1990@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

patch tool: await keyword corruption in TypeScript/TSX files [Bug]: Redaction incorrectly masks lowercase Python variable assignments

2 participants

@gnanam1990 @teknium1