Skip to content

feat: Support frontmatter in docs#344

Merged
AndrewSouthpaw merged 35 commits into
thlorenz:masterfrom
thompson-tomo:feat/#279_frontMatter
Jun 12, 2026
Merged

feat: Support frontmatter in docs#344
AndrewSouthpaw merged 35 commits into
thlorenz:masterfrom
thompson-tomo:feat/#279_frontMatter

Conversation

@thompson-tomo

Copy link
Copy Markdown
Contributor

Closes #279

Insert toc after front matter content to ensure front matter remains valid.

@AndrewSouthpaw AndrewSouthpaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Looks like it crashes for an empty file
  • Crashes for a file with only the YAML frontmatter and no other content
  • Same for JSON
  • Probably add a test for re-running on a TOC'ed frontmatter file?
  • Test case with processAll?
  • The startsWith/endsWith check on .raw breaks if the closing delimiter has trailing whitespace, e.g. "+++\ntitle: test\n+++ \n"

Comment thread lib/transform.js Outdated
@thompson-tomo

Copy link
Copy Markdown
Contributor Author

@AndrewSouthpaw i believe all the additional scenarios are being catered for now with the additional tests as well as robustness improvements

@AndrewSouthpaw AndrewSouthpaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two small notes, looks just about ready!

Comment thread lib/transform.js Outdated
Comment thread lib/transform.js Outdated
@thompson-tomo thompson-tomo force-pushed the feat/#279_frontMatter branch from d106897 to 49b8b3f Compare April 25, 2026 03:03

@AndrewSouthpaw AndrewSouthpaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! It looks like there's one regression worth addressing here. The frontmatter doc/body split in check() above assumes the
closing delimiter is followed by exactly one blank line:

var pos = md.indexOf(tag,3) + 3 + 2 * eol.length;

With anything other than one blank line, pos is wrong (here, 0 blank lines drags the first char of the next line into the doc segment).

Separately, md.substring(0, 3) matches any 3-char prefix in ['---', ';;;', '+++'], so a horizontal-rule "---" at the top of a doc is misclassified as frontmatter and falls into the same broken path. Both should round-trip once the helper scans for the next non-blank line after a verified closing tag.

A couple sample checks, which both fail right now:

// 0 blank lines after the closing delimiter
check(
  [ '---'
  , 'title: Yaml'
  , '---'
  , '# H'
  , 'body'
  ].join('\n')
, [ '**Table of Contents**  *generated with [DocToc](https://github.com/thlorenz/doctoc)*\n\n'
  , '- [H](#h)\n\n\n'
  ].join('')
)

// "---" at the top is a horizontal rule, not frontmatter — helper false-positive
check(
  [ '---'
  , ''
  , '# Real Heading'
  , 'body'
  ].join('\n')
, [ '**Table of Contents**  *generated with [DocToc](https://github.com/thlorenz/doctoc)*\n\n'
  , '- [Real Heading](#real-heading)\n\n\n'
  ].join('')
)

I was inclined to let it be a follow-up, as I think it's maybe unlikely for docs to have a leading --- without it being frontmatter... But it would be quite unfortunate if they did. Let's do one more round and address this as in scope to the PR.

@thompson-tomo

Copy link
Copy Markdown
Contributor Author

Thanks @AndrewSouthpawfor picking up those cases. I have added those checks and am ensuring they are passing.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for documents that start with front matter by ensuring the generated TOC is inserted after the front matter region (keeping the front matter block valid), and expands the test suite to cover YAML/TOML/JSON-style delimiters.

Changes:

  • Adjust TOC insertion and header scanning start positions to account for detected front matter.
  • Add a dedicated front matter test suite and fixtures (yaml/toml/json/invalid/blank/existing-TOC cases).
  • Update the shared transform test helper to build expected docs with TOC placed after front matter.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

File Description
lib/transform.js Detect front matter and shift insertion/header-processing offsets to begin after it.
test/transform.js Update test helper to build expected output with TOC after front matter; add a couple front matter-related checks.
test/transform-frontmatter.js New tests validating TOC placement with multiple front matter styles and edge cases.
test/fixtures/readme-frontmatter-*.md New fixtures used by the new front matter tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/transform.js Outdated
Comment thread lib/transform.js
Comment thread test/transform-frontmatter.js
@thompson-tomo

thompson-tomo commented Apr 26, 2026

Copy link
Copy Markdown
Contributor Author

Good news with textlint/textlint#2015 we can natively detect know if json/toml, hence ci broken for package to be updated.

@thompson-tomo thompson-tomo force-pushed the feat/#279_frontMatter branch from cc1308e to e9e8b17 Compare April 27, 2026 04:09
@thompson-tomo

Copy link
Copy Markdown
Contributor Author

@AndrewSouthpaw could you please give this another look over.

@thompson-tomo

Copy link
Copy Markdown
Contributor Author

@AndrewSouthpaw could you take a look as this should now be good to merge along with the couple of pr's I have open.

@AndrewSouthpaw AndrewSouthpaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

almost there, I think there's another edge case where a stray \r\n can lurk:

test('\ngiven a file with CRLF line endings and yaml frontmatter', function (t) {
  var eol = '\r\n';
  var content = ['---', 'title: Yaml Front Matter', '---', '', '# Heading', '', 'Your regular Markdown content follows...', ''].join(eol);
  var res = transform(content);

  t.same(
      res.data
    , [ '---',
        'title: Yaml Front Matter',
        '---',
        '',
        '<!-- START doctoc generated TOC please keep comment here to allow auto update -->',
        '<!-- DON\'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->',
        '**Table of Contents**  *generated with [DocToc](https://github.com/thlorenz/doctoc)*',
        '',
        '- [Heading](#heading)',
        '',
        '<!-- END doctoc generated TOC please keep comment here to allow auto update -->',
        '',
        '# Heading',
        '',
        'Your regular Markdown content follows...',
        '' ].join(eol)
    , 'generates correct toc preserving CRLF line endings with frontmatter'
  )

  t.end()
})

This test checks the transformation of content with CRLF line endings and YAML frontmatter, ensuring the correct generation of the table of contents.
@thompson-tomo

thompson-tomo commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

There you are @AndrewSouthpaw have added the provided test & gotten it passing. Turns out there was 2 scenarios which were still assuming \n with both now fixed.

@AndrewSouthpaw AndrewSouthpaw merged commit 59dcae9 into thlorenz:master Jun 12, 2026
5 checks passed
@thompson-tomo thompson-tomo deleted the feat/#279_frontMatter branch June 13, 2026 01:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Increased compatability when front matter is used

3 participants