fix: prevent heading and def tokens from greedily capturing multiple newlines by Puxhkar · Pull Request #3925 · markedjs/marked

Puxhkar · 2026-03-23T07:57:05Z

…nitions

Marked version:

Markdown flavor: Markdown.pl|CommonMark|GitHub Flavored Markdown|n/a

Description

Fixes #### (if fixing a known issue; otherwise, describe issue using the following format)

Contributor

Test(s) exist to ensure functionality and minimize regression (if no tests added, list tests covering this PR); or,
no tests required for this PR.
If submitting new feature, it has been documented in the appropriate places.

Committer

In most cases, this should be a different person than the contributor.

CI is green (no forced merge required).
Squash and Merge PR following conventional commit guidelines.

…nitions

vercel · 2026-03-23T07:57:11Z

@servis is attempting to deploy a commit to the MarkedJS Team on Vercel.

A member of the Team first needs to authorize it.

gemini-code-assist · 2026-03-23T07:58:17Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on refining the tokenization process in marked.js by preventing heading and definition tokens from capturing excessive newlines. This adjustment ensures more accurate token representation and improves the overall parsing behavior of the library.

Highlights

Newline Handling: This PR addresses an issue where heading and definition tokens were greedily capturing multiple newlines.
Raw Token Values: The changes ensure that the raw property of heading and def tokens no longer includes trailing newlines.
Test Updates: The unit tests were updated to reflect the change in the raw property of heading tokens.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist · 2026-03-23T08:03:25Z

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

vercel · 2026-03-26T04:29:20Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
marked-website	Ready	Preview, Comment	Mar 26, 2026 4:29am

UziTech · 2026-03-26T04:30:44Z

I will have to test this against a couple extensions. This could be a breaking change.

Puxhkar · 2026-03-26T17:53:12Z

I will have to test this against a couple extensions. This could be a breaking change.

Good point, thanks for calling that out.

I’ll test this with a few extensions and edge cases (especially ones that rely on raw and newline handling) to make sure nothing breaks. If I find any issues, I can either adjust the implementation or document it properly as a breaking change.

Let me know if there are any specific extensions you’d like me to check as well.

UziTech · 2026-03-29T19:42:04Z

I looked at a few extensions that use the tokens and I don't see any that would break due to this change, but I still think we should consider this a breaking change since it is not really a fix.

If we want to be consistent we should also make html blocks and indented code blocks do the same thing.

UziTech · 2026-04-06T07:03:58Z

I created #3939 to include html blocks and indented code blocks. We will try to get this out for the next major change.

UziTech

Thanks for fixing this 💯

Puxhkar · 2026-04-06T12:10:21Z

Thanks for fixing this 💯

can you please merge the PR

# [18.0.0](v17.0.6...v18.0.0) (2026-04-07) ### Bug Fixes * Bump typescript from 5.9.3 to 6.0.2 ([#3934](#3934)) ([e8efc51](e8efc51)) * prevent GFM table tokens from greedily capturing trailing newlines ([#3926](#3926)) ([40f2665](40f2665)) * prevent heading and def tokens from greedily capturing multiple newlines ([#3925](#3925)) ([b379e3e](b379e3e)) * trim blank lines from block tokens ([#3939](#3939)) ([b70895f](b70895f)) ### BREAKING CHANGES * trim trailing blank lines from block tokens * update Typescript to v6

# [18.0.0](markedjs/marked@v17.0.6...v18.0.0) (2026-04-07) ### Bug Fixes * Bump typescript from 5.9.3 to 6.0.2 ([#3934](markedjs/marked#3934)) ([44b1b93](markedjs/marked@44b1b93)) * prevent GFM table tokens from greedily capturing trailing newlines ([#3926](markedjs/marked#3926)) ([771f5c8](markedjs/marked@771f5c8)) * prevent heading and def tokens from greedily capturing multiple newlines ([#3925](markedjs/marked#3925)) ([4138f72](markedjs/marked@4138f72)) * trim blank lines from block tokens ([#3939](markedjs/marked#3939)) ([fbebb3b](markedjs/marked@fbebb3b)) ### BREAKING CHANGES * trim trailing blank lines from block tokens * update Typescript to v6

chocolateboy · 2026-06-16T09:28:08Z

I looked at a few extensions that use the tokens and I don't see any that would break due to this change, but I still think we should consider this a breaking change since it is not really a fix.

FYI: this is a breaking change for toc-md and its (slightly less unmaintained) derivatives:

Details

Input

<!-- TOC -->

  h1
===

h2
--

### h3

Expected

<!-- TOC -->

- [  h1](#--h1)
  - [h2](#h2)
    - [h3](#h3)

<!-- TOC END -->

  h1
===

h2
--

### h3

Actual

<!-- TOC -->

- [h1](#h1)
  - [h2](#h2)
    - [h3](#h3)

<!-- TOC END -->

  h1
===

h2
--

### h3

fix: correctly handle multiple trailing newlines in headings and defi…

a91e9b2

…nitions

vercel Bot deployed to Preview March 26, 2026 04:29 View deployment

UziTech added proposal Breaking Change and removed proposal labels Apr 4, 2026

UziTech mentioned this pull request Apr 6, 2026

BREAKING CHANGE: trim blank lines from block tokens #3939

Merged

5 tasks

UziTech approved these changes Apr 6, 2026

View reviewed changes

UziTech requested review from calculuschild and styfle April 6, 2026 07:19

styfle approved these changes Apr 6, 2026

View reviewed changes

UziTech merged commit b379e3e into markedjs:master Apr 7, 2026
8 checks passed

Uh oh!

Conversation

Puxhkar commented Mar 23, 2026

Description

Contributor

Committer

Uh oh!

vercel Bot commented Mar 23, 2026

Uh oh!

gemini-code-assist Bot commented Mar 23, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot commented Mar 23, 2026

Uh oh!

vercel Bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

UziTech commented Mar 26, 2026

Uh oh!

Puxhkar commented Mar 26, 2026

Uh oh!

UziTech commented Mar 29, 2026

Uh oh!

UziTech commented Apr 6, 2026

Uh oh!

UziTech left a comment

Choose a reason for hiding this comment

Uh oh!

Puxhkar commented Apr 6, 2026

Uh oh!

Uh oh!

chocolateboy commented Jun 16, 2026

Input

Expected

Actual

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vercel Bot commented Mar 26, 2026 •

edited

Loading