Skip to content

fix(skills,memory): handle BOM, folded lines, and quoted values in frontmatter#690

Merged
esengine merged 2 commits into
mainfrom
fix/frontmatter-parser
May 12, 2026
Merged

fix(skills,memory): handle BOM, folded lines, and quoted values in frontmatter#690
esengine merged 2 commits into
mainfrom
fix/frontmatter-parser

Conversation

@esengine

Copy link
Copy Markdown
Owner

Summary

The three local copies of the frontmatter parser (src/skills.ts, src/memory/user.ts, src/server/api/skills.ts) shared three defects:

  1. UTF-8 BOM not strippedlines[0] !== "---" failed for any file saved with a BOM, so the whole frontmatter block was treated as body.
  2. Folded / continuation lines dropped — a wrapped description: like
    description: first line
      second line
    kept only first line because the continuation didn't match the key: value regex.
  3. Quotes leaked into valuesdescription: "text" was stored as "text", so the literal quotes showed up in the dashboard and the skills index.

Consolidates the three copies into one parser at src/frontmatter.ts:

  • strips a leading \uFEFF before checking for the opening ---
  • folds indented non-key lines onto the preceding key with a single-space join, resetting scope on blank lines
  • unwraps matching "…" / '…' after folding (so quoted multi-line values work too)

Three call sites now import the shared parser. parseFrontmatterDescription in the dashboard API is a two-line wrapper that picks data.description.

Closes #678

Test plan

  • tests/frontmatter.test.ts — 11 cases covering BOM, folded onto inline value, folded onto empty value, double-quote strip, single-quote strip, mismatched-quote preservation, missing delimiters, CRLF endings, blank-line scope reset
  • npx vitest run — 2639 passed / 2 skipped
  • npm run lint — clean
  • npm run typecheck — clean

…ontmatter

The three local copies of the frontmatter parser all skipped files with a
UTF-8 BOM, dropped indented continuation lines, and left wrapping quotes
in description values. Consolidates them into one parser that strips the
BOM, folds indented continuations onto the preceding key, and unwraps
matching single/double quotes after folding.

Closes #678
Comment thread src/frontmatter.ts Fixed
Comment thread src/frontmatter.ts Fixed
Comment thread src/frontmatter.ts Fixed
The KEY_RE regex matches `__proto__` / `constructor` / `prototype`, so a
bracket write `data[key] = value` could mutate Object.prototype. Switch
the accumulator to a Map (no prototype to pollute), filter forbidden
keys at both the set and materialization sites, and return a
null-prototype object. Resolves CodeQL js/prototype-pollution-utility.
@esengine esengine merged commit 140f97f into main May 12, 2026
3 checks passed
@esengine esengine deleted the fix/frontmatter-parser branch May 12, 2026 07:26
esengine added a commit that referenced this pull request May 12, 2026
…il, CardStream fix (#705)

npm-only release. The Tauri desktop source is in the repo and the CLI
subcommand works, but installer bundles for macOS / Windows / Linux
don't ship this round (separate release once signing's settled).

Highlights:
- Tauri desktop client with multi-tab concurrent runtimes (#689)
  plus a near-full polish pass: wallet balance, version chip, active-
  plan rail, abortable pause-gates, edit-gate pill, en + zh-CN i18n,
  shared pause-policy module dedup'd with the CLI TUI (#701)
- checkpoint API + git-changes panel in the embedded dashboard (#682)
- outside-sandbox file access approval modal (#696)
- MCP loading pill + readiness gate on tool dispatch (#687)
- escalate-after flag for flash → pro threshold (#699)

Fixes:
- CardStream Maximum-update-depth crash, quantize window so boundary
  cards stop oscillating (#700, #702)
- `reasonix code` bridges config key to env + lazy subagent client so
  fresh installs can reach the setup wizard (#703)
- pinned-mode scroll shrinks coalesced (#666), generic CSI key decode
  (#692), shell-confirm preview clamp (#691), frontmatter BOM/folded
  lines (#690), MCP error classification (#688), and more
ChasLui pushed a commit to ChasLui/DeepSeek-Reasonix that referenced this pull request May 23, 2026
…ontmatter (esengine#690)

* fix(skills,memory): handle BOM, folded lines, and quoted values in frontmatter

The three local copies of the frontmatter parser all skipped files with a
UTF-8 BOM, dropped indented continuation lines, and left wrapping quotes
in description values. Consolidates them into one parser that strips the
BOM, folds indented continuations onto the preceding key, and unwraps
matching single/double quotes after folding.

Closes esengine#678

* fix(frontmatter): guard against prototype-pollution keys

The KEY_RE regex matches `__proto__` / `constructor` / `prototype`, so a
bracket write `data[key] = value` could mutate Object.prototype. Switch
the accumulator to a Map (no prototype to pollute), filter forbidden
keys at both the set and materialization sites, and return a
null-prototype object. Resolves CodeQL js/prototype-pollution-utility.
ChasLui pushed a commit to ChasLui/DeepSeek-Reasonix that referenced this pull request May 23, 2026
…il, CardStream fix (esengine#705)

npm-only release. The Tauri desktop source is in the repo and the CLI
subcommand works, but installer bundles for macOS / Windows / Linux
don't ship this round (separate release once signing's settled).

Highlights:
- Tauri desktop client with multi-tab concurrent runtimes (esengine#689)
  plus a near-full polish pass: wallet balance, version chip, active-
  plan rail, abortable pause-gates, edit-gate pill, en + zh-CN i18n,
  shared pause-policy module dedup'd with the CLI TUI (esengine#701)
- checkpoint API + git-changes panel in the embedded dashboard (esengine#682)
- outside-sandbox file access approval modal (esengine#696)
- MCP loading pill + readiness gate on tool dispatch (esengine#687)
- escalate-after flag for flash → pro threshold (esengine#699)

Fixes:
- CardStream Maximum-update-depth crash, quantize window so boundary
  cards stop oscillating (esengine#700, esengine#702)
- `reasonix code` bridges config key to env + lazy subagent client so
  fresh installs can reach the setup wizard (esengine#703)
- pinned-mode scroll shrinks coalesced (esengine#666), generic CSI key decode
  (esengine#692), shell-confirm preview clamp (esengine#691), frontmatter BOM/folded
  lines (esengine#690), MCP error classification (esengine#688), and more
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

parseFrontmatter(读取 skill 用)和 parseFrontmatter2(读取 memory 用)存在三个缺陷

2 participants