Skip to content

Unicode newline characters are handled incorrectly in Markdown preview #166015

@miyaokamarina

Description

@miyaokamarina

When rendering Markdown preview, Unicode newline characters U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR are replaced with an empty string at the source preprocessing stage.

This behavior was introduced in #63936 to fix #63749, but it breaks the document in multiple ways:

  • double-click word selection does not work as expected,
  • screen readers may announce the text incorrectly,
  • line wrapping and hyphenation does not work correctly,
  • markdown-it plugins that rely on \b or \s may work incorrectly,
  • even linkify fails:
    https://foo.bar/baz<LS>Lorem ipsum
    (Expected link: https://foo.bar/baz)
    (Received link: https://foo.bar/bazLorem)
    
  • and so on.

Since then it was fixed at the markdown-it side, and #63936 is no longer needed:

It would be better to revert this change, and keep the Markdown source as is before feeding it to the parser.


Steps to reproduce:

  1. Create a Markdown document like this:
    https://foo.bar/baz
Lorem ipsum. Lorem
ipsum.
    https://foo.bar/baz
Lorem ipsum. Lorem
ipsum.
  2. Open preview:
    • Expected result: LS and PS are displayed as is, links end at Unicode newlines.
    • Actual result: no Unicode newline characters displayed, links also consume the immediately following word.

Does this issue occur when all extensions are disabled?: Yes

  • VS Code Version:
    Version: 1.73.0 (system setup)
    Commit: 8fa188b2b301d36553cbc9ce1b0a146ccb93351f
    Date: 2022-11-01T15:34:06.111Z
    Electron: 19.0.17
    Chromium: 102.0.5005.167
    Node.js: 16.14.2
    V8: 10.2.154.15-electron.0
    OS: Windows_NT x64 10.0.22000
    Sandboxed: No
    
  • OS Version: Windows_NT x64 10.0.22000

Metadata

Metadata

Assignees

Labels

author-verification-requestedIssues potentially verifiable by issue authorbugIssue identified by VS Code Team member as probable buginsiders-releasedPatch has been released in VS Code InsidersmarkdownMarkdown support issuesverifiedVerification succeeded

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions