fix(lexer): Only allow horizontal whitespace in frontmatter

In writing up the reference for frontmatter, I realized that we probably
shouldn't be accepting Unicode Line Ending characters between the code
fence and infostring or trailing after the infostring or a code fence.

In digging into the unicode specification we use for Whitespace, it
divides it up into categories, so I'm deferring to what it says for
horizontal whitespace for what should be used within a line.

Note, I am leaving out support for Unicode Default Ignorable characters.
I figure that can be discussed outside of this change within the
reference and tracking issue.

Fixes rust-lang#145971

Frontmatter tracking issue: rust-lang#136889

Rollup of 8 pull requests

Successful merges:

 - #139113 (unstable book: in a sanitizer example, check the code)
 - #145279 (Constify conversion traits (part 1))
 - #145414 (unicode-table-generator refactors)
 - #145823 (editorconfig: don't use nonexistent syntax)
 - #145944 (std: Start supporting WASIp2 natively )
 - #145961 (resolve: Avoid a regression from splitting prelude into two scopes)
 - #146032 (Explicity disable LSX feature for `loongarch64-unknown-none` target)
 - #146106 (fix(lexer): Only allow horizontal whitespace in frontmatter )

r? `@ghost`
`@rustbot` modify labels: rollup

Rollup of 8 pull requests

Successful merges:

 - #139113 (unstable book: in a sanitizer example, check the code)
 - #145279 (Constify conversion traits (part 1))
 - #145414 (unicode-table-generator refactors)
 - #145823 (editorconfig: don't use nonexistent syntax)
 - #145944 (std: Start supporting WASIp2 natively )
 - #145961 (resolve: Avoid a regression from splitting prelude into two scopes)
 - #146032 (Explicity disable LSX feature for `loongarch64-unknown-none` target)
 - #146106 (fix(lexer): Only allow horizontal whitespace in frontmatter )

r? `@ghost`
`@rustbot` modify labels: rollup

fix(lexer): Only allow horizontal whitespace in frontmatter

In writing up the reference for frontmatter, I realized that we probably
shouldn't be accepting Unicode Line Ending characters between the code
fence and infostring or trailing after the infostring or a code fence.

In digging into the unicode specification we use for Whitespace, it
divides it up into categories, so I'm deferring to what it says for
horizontal whitespace for what should be used within a line.

Note, I am leaving out support for Unicode Default Ignorable characters.
I figure that can be discussed outside of this change within the
reference and tracking issue.

Fixes rust-lang#145971

Frontmatter tracking issue: rust-lang#136889

Rollup of 9 pull requests

Successful merges:

 - #145279 (Constify conversion traits (part 1))
 - #145414 (unicode-table-generator refactors)
 - #145823 (editorconfig: don't use nonexistent syntax)
 - #145944 (std: Start supporting WASIp2 natively )
 - #145961 (resolve: Avoid a regression from splitting prelude into two scopes)
 - #146032 (Explicity disable LSX feature for `loongarch64-unknown-none` target)
 - #146106 (fix(lexer): Only allow horizontal whitespace in frontmatter )
 - #146112 (don't uppercase error messages)
 - #146154 (CI: rfl: move job forward to Linux v6.17-rc3 plus 2 commits)

r? `@ghost`
`@rustbot` modify labels: rollup

Rollup of 8 pull requests

Successful merges:

 - #145279 (Constify conversion traits (part 1))
 - #145414 (unicode-table-generator refactors)
 - #145823 (editorconfig: don't use nonexistent syntax)
 - #145944 (std: Start supporting WASIp2 natively )
 - #145961 (resolve: Avoid a regression from splitting prelude into two scopes)
 - #146032 (Explicity disable LSX feature for `loongarch64-unknown-none` target)
 - #146106 (fix(lexer): Only allow horizontal whitespace in frontmatter )
 - #146154 (CI: rfl: move job forward to Linux v6.17-rc3 plus 2 commits)

r? `@ghost`
`@rustbot` modify labels: rollup

Rollup merge of #146106 - epage:whitespace, r=fee1-dead

fix(lexer): Only allow horizontal whitespace in frontmatter

In writing up the reference for frontmatter, I realized that we probably
shouldn't be accepting Unicode Line Ending characters between the code
fence and infostring or trailing after the infostring or a code fence.

In digging into the unicode specification we use for Whitespace, it
divides it up into categories, so I'm deferring to what it says for
horizontal whitespace for what should be used within a line.

Note, I am leaving out support for Unicode Default Ignorable characters.
I figure that can be discussed outside of this change within the
reference and tracking issue.

Fixes #145971

Frontmatter tracking issue: #136889

This is to align with rust-lang/rust#146106.

While no tests were affected by this,
I definitely broke a lot of tests when making this change.

This is to align with rust-lang/rust#146106.

While no tests were affected by this,
I definitely broke a lot of tests when making this change.

### What does this PR try to resolve?

This is to align with rust-lang/rust#146106 and is part of #12207.

### How to test and review this PR?

While no tests were affected by this,
I definitely broke a lot of tests when making this change.

The comment on `is_horizontal_whitespace` says "This is
Pattern_White_Space", but the function matches only tab (U+0009) and
space (U+0020) -- two of the eleven `Pattern_White_Space` code points.
This has been the case since rust-lang#146106, which narrowed the
set from full `Pattern_White_Space` to the horizontal subset.

The correct characterization is that this is the horizontal space
subset of `Pattern_White_Space`, as categorized by UAX 31,
Section 4.1, which partitions `Pattern_White_Space` into line endings,
ignorable format controls, and horizontal space.

…tespace-comment, r=Kivooeo,epage

Fix comment on `is_horizontal_whitespace`

The comment on `is_horizontal_whitespace` says "This is Pattern_White_Space", but the function matches only tab (U+0009) and space (U+0020) -- two of the eleven `Pattern_White_Space` code points. This has been the case since rust-lang#146106, which narrowed the set from full `Pattern_White_Space` to the horizontal subset.

The correct characterization is that this is the horizontal space subset of `Pattern_White_Space`, as categorized by UAX 31, Section 4.1, which partitions `Pattern_White_Space` into line endings, ignorable format controls, and horizontal space.

r? fee1-dead (reviewer on rust-lang#146106)

cc @ehuss @epage

Rollup merge of #153385 - traviscross:TC/fix-frontmatter-whitespace-comment, r=Kivooeo,epage

Fix comment on `is_horizontal_whitespace`

The comment on `is_horizontal_whitespace` says "This is Pattern_White_Space", but the function matches only tab (U+0009) and space (U+0020) -- two of the eleven `Pattern_White_Space` code points. This has been the case since #146106, which narrowed the set from full `Pattern_White_Space` to the horizontal subset.

The correct characterization is that this is the horizontal space subset of `Pattern_White_Space`, as categorized by UAX 31, Section 4.1, which partitions `Pattern_White_Space` into line endings, ignorable format controls, and horizontal space.

r? fee1-dead (reviewer on #146106)

cc @ehuss @epage