LibUnicode: Update to Unicode version 15.0.0 by trflynn89 · Pull Request #15230 · SerenityOS/serenity

trflynn89 · 2022-09-13T21:37:57Z

https://unicode.org/versions/Unicode15.0.0/

trflynn89 · 2022-09-13T21:42:23Z

Draft for now because:

    Diff Tests:
         -63 ✅   +63 ❌

These are the tests under test/built-ins/RegExp/property-escapes/generated. Code point properties apparently changed quite a bit.

Full diff

Diff Tests:
    test/built-ins/RegExp/property-escapes/generated/Alphabetic.js                               ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Assigned.js                                 ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Case_Ignorable.js                           ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Cased.js                                    ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Changes_When_NFKC_Casefolded.js             ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Diacritic.js                                ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Emoji.js                                    ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Emoji_Modifier_Base.js                      ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Emoji_Presentation.js                       ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Cased_Letter.js          ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Decimal_Number.js        ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Format.js                ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Letter.js                ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Lowercase_Letter.js      ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Mark.js                  ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Modifier_Letter.js       ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Nonspacing_Mark.js       ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Number.js                ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Other.js                 ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Other_Letter.js          ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Other_Number.js          ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Other_Punctuation.js     ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Other_Symbol.js          ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Punctuation.js           ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Spacing_Mark.js          ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Symbol.js                ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Unassigned.js            ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Grapheme_Base.js                            ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Grapheme_Extend.js                          ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/ID_Continue.js                              ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/ID_Start.js                                 ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Ideographic.js                              ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Lowercase.js                                ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_-_Arabic.js                          ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_-_Common.js                          ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_-_Cyrillic.js                        ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_-_Devanagari.js                      ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_-_Egyptian_Hieroglyphs.js            ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_-_Han.js                             ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_-_Hiragana.js                        ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_-_Kannada.js                         ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_-_Katakana.js                        ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_-_Khojki.js                          ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_-_Lao.js                             ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_-_Latin.js                           ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Arabic.js               ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Common.js               ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Cyrillic.js             ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Devanagari.js           ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Egyptian_Hieroglyphs.js ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Han.js                  ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Hiragana.js             ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Kannada.js              ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Katakana.js             ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Khojki.js               ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Lao.js                  ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Latin.js                ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Sentence_Terminal.js                        ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Soft_Dotted.js                              ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Terminal_Punctuation.js                     ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/Unified_Ideograph.js                        ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/XID_Continue.js                             ✅ -> ❌
    test/built-ins/RegExp/property-escapes/generated/XID_Start.js                                ✅ -> ❌

davidot · 2022-09-21T09:38:29Z

New Unicode 15 tests have been merged in test262:

Summary:
    New Tests:
        +12 ❌   
    Diff Tests:
         -63 ✅   +63 ❌

The generators had a manual implementation when Core::Stream did not have a read_all method.

https://unicode.org/versions/Unicode15.0.0/

trflynn89 · 2022-09-21T12:01:20Z

All tests pass!

    Diff Tests:
        +75 ✅    -75 ❌

Test diff

Diff Tests:
    test/built-ins/RegExp/property-escapes/generated/Alphabetic.js                               ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Assigned.js                                 ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Case_Ignorable.js                           ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Cased.js                                    ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Changes_When_NFKC_Casefolded.js             ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Diacritic.js                                ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Emoji.js                                    ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Emoji_Modifier_Base.js                      ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Emoji_Presentation.js                       ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Cased_Letter.js          ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Decimal_Number.js        ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Format.js                ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Letter.js                ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Lowercase_Letter.js      ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Mark.js                  ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Modifier_Letter.js       ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Nonspacing_Mark.js       ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Number.js                ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Other.js                 ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Other_Letter.js          ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Other_Number.js          ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Other_Punctuation.js     ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Other_Symbol.js          ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Punctuation.js           ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Spacing_Mark.js          ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Symbol.js                ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/General_Category_-_Unassigned.js            ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Grapheme_Base.js                            ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Grapheme_Extend.js                          ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/ID_Continue.js                              ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/ID_Start.js                                 ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Ideographic.js                              ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Lowercase.js                                ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Arabic.js                          ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Common.js                          ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Cyrillic.js                        ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Devanagari.js                      ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Egyptian_Hieroglyphs.js            ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Han.js                             ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Hiragana.js                        ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Kannada.js                         ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Katakana.js                        ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Kawi.js                            ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Khojki.js                          ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Lao.js                             ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Latin.js                           ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_-_Nag_Mundari.js                     ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Arabic.js               ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Common.js               ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Cyrillic.js             ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Devanagari.js           ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Egyptian_Hieroglyphs.js ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Han.js                  ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Hiragana.js             ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Kannada.js              ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Katakana.js             ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Kawi.js                 ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Khojki.js               ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Lao.js                  ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Latin.js                ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Nag_Mundari.js          ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Sentence_Terminal.js                        ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Soft_Dotted.js                              ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Terminal_Punctuation.js                     ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/Unified_Ideograph.js                        ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/XID_Continue.js                             ❌ -> ✅
    test/built-ins/RegExp/property-escapes/generated/XID_Start.js                                ❌ -> ✅
    test/language/identifiers/part-unicode-15.0.0-class-escaped.js                               ❌ -> ✅
    test/language/identifiers/part-unicode-15.0.0-class.js                                       ❌ -> ✅
    test/language/identifiers/part-unicode-15.0.0-escaped.js                                     ❌ -> ✅
    test/language/identifiers/part-unicode-15.0.0.js                                             ❌ -> ✅
    test/language/identifiers/start-unicode-15.0.0-class-escaped.js                              ❌ -> ✅
    test/language/identifiers/start-unicode-15.0.0-class.js                                      ❌ -> ✅
    test/language/identifiers/start-unicode-15.0.0-escaped.js                                    ❌ -> ✅
    test/language/identifiers/start-unicode-15.0.0.js                                            ❌ -> ✅

trflynn89 marked this pull request as draft September 13, 2022 21:38

trflynn89 force-pushed the unicode_15 branch 2 times, most recently from 2ed0425 to 506bf85 Compare September 14, 2022 13:01

trflynn89 mentioned this pull request Sep 14, 2022

Update RegExp \p{…} tests per Unicode 15 tc39/test262#3663

Merged

trflynn89 added 2 commits September 21, 2022 07:53

LibUnicode: Use recently added Core::Stream::read_all in code generators

eb51d3c

The generators had a manual implementation when Core::Stream did not have a read_all method.

LibUnicode: Update to Unicode version 15.0.0

9337e2c

https://unicode.org/versions/Unicode15.0.0/

trflynn89 force-pushed the unicode_15 branch from 506bf85 to 9337e2c Compare September 21, 2022 12:00

trflynn89 marked this pull request as ready for review September 21, 2022 12:00

AtkinsSJ merged commit 51854e3 into SerenityOS:master Sep 21, 2022

trflynn89 deleted the unicode_15 branch September 21, 2022 13:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LibUnicode: Update to Unicode version 15.0.0#15230

LibUnicode: Update to Unicode version 15.0.0#15230
AtkinsSJ merged 2 commits into
SerenityOS:masterfrom
trflynn89:unicode_15

trflynn89 commented Sep 13, 2022

Uh oh!

trflynn89 commented Sep 13, 2022

Uh oh!

davidot commented Sep 21, 2022

Uh oh!

trflynn89 commented Sep 21, 2022 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

trflynn89 commented Sep 13, 2022

Uh oh!

trflynn89 commented Sep 13, 2022

Uh oh!

davidot commented Sep 21, 2022

Uh oh!

trflynn89 commented Sep 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

trflynn89 commented Sep 21, 2022 •

edited

Loading