Add Gleam lexer by darenc · Pull Request #2662 · pygments/pygments

darenc · 2024-03-13T17:19:06Z

Hi, this PR adds a lexer for Gleam, which just released version 1.0.0.

I've run tests and things look good to me. However, I'm not a Python person and haven't tried writing a lexer before, particularly not one for Pygments, so I'm happy to take feedback and suggestions for improvements.

There is some example HTML at https://red-rose.com/gleam.html that I generated with pygmentize and then modified to add the lightbulb style (slightly modified for Name colours). This is the same large example file that's included with the PR.

Cheers,
Daren

jeanas · 2024-03-13T17:30:07Z

(Pushed a commit to fix line endings, can you configure Git's core.autocrlf for the future? Thanks.)

darenc · 2024-03-13T17:42:15Z

(Pushed a commit to fix line endings, can you configure Git's core.autocrlf for the future? Thanks.)

Thank you, I thought I'd fixed them. Sorry for the inconvenience

… gleam-lexer

pygments/lexers/gleam.py

jeanas · 2024-03-29T11:51:38Z

pygments/lexers/gleam.py

+            (r'[a-zA-Z_/]\w*', Name),
+
+            # numbers
+            (r'(\d+((_\d+)+)?\.(?!\.)(\d+((_\d+)+)?)?|\.\d+((_\d+)+)?)([eEf][+-]?[0-9]+)?', Number.Float),


Suggested change

(r'(\d+((_\d+)+)?\.(?!\.)(\d+((_\d+)+)?)?|\.\d+((_\d+)+)?)([eEf][+-]?[0-9]+)?', Number.Float),

(r'(\d+(_\d+)*\.(?!\.)(\d+(_\d+)*)?|\.\d+(_\d+)*)([eEf][+-]?[0-9]+)?', Number.Float),

(X+)? is just X*.

jeanas · 2024-03-29T11:52:24Z

pygments/lexers/gleam.py

+
+            # numbers
+            (r'(\d+((_\d+)+)?\.(?!\.)(\d+((_\d+)+)?)?|\.\d+((_\d+)+)?)([eEf][+-]?[0-9]+)?', Number.Float),
+            (r'\d+((_\d+)+)?[eEf][+-]?[0-9]+', Number.Float),


Suggested change

(r'\d+((_\d+)+)?[eEf][+-]?[0-9]+', Number.Float),

(r'\d+(_\d+)*[eEf][+-]?[0-9]+', Number.Float),

jeanas · 2024-03-29T11:56:09Z

pygments/lexers/gleam.py

+            # numbers
+            (r'(\d+((_\d+)+)?\.(?!\.)(\d+((_\d+)+)?)?|\.\d+((_\d+)+)?)([eEf][+-]?[0-9]+)?', Number.Float),
+            (r'\d+((_\d+)+)?[eEf][+-]?[0-9]+', Number.Float),
+            (r'0[xX][a-fA-F0-9]+((_[a-fA-F0-9]+)+)?(\.([a-fA-F0-9]+((_[a-fA-F0-9]+)+)?)?)?p[+-]?\d+', Number.Float),


Suggested change

(r'0[xX][a-fA-F0-9]+((_[a-fA-F0-9]+)+)?(\.([a-fA-F0-9]+((_[a-fA-F0-9]+)+)?)?)?p[+-]?\d+', Number.Float),

(r'0[xX][a-fA-F0-9]+(_[a-fA-F0-9]+)*(\.([a-fA-F0-9]+(_[a-fA-F0-9]+)*)?)?p[+-]?\d+', Number.Float),

jeanas · 2024-03-29T11:56:30Z

pygments/lexers/gleam.py

+            (r'(\d+((_\d+)+)?\.(?!\.)(\d+((_\d+)+)?)?|\.\d+((_\d+)+)?)([eEf][+-]?[0-9]+)?', Number.Float),
+            (r'\d+((_\d+)+)?[eEf][+-]?[0-9]+', Number.Float),
+            (r'0[xX][a-fA-F0-9]+((_[a-fA-F0-9]+)+)?(\.([a-fA-F0-9]+((_[a-fA-F0-9]+)+)?)?)?p[+-]?\d+', Number.Float),
+            (r'0[bB][01]+((_[01]+)+)?', Number.Bin),


Suggested change

(r'0[bB][01]+((_[01]+)+)?', Number.Bin),

(r'0[bB][01]+(_[01]+)*', Number.Bin),

jeanas · 2024-03-29T11:56:42Z

pygments/lexers/gleam.py

+            (r'\d+((_\d+)+)?[eEf][+-]?[0-9]+', Number.Float),
+            (r'0[xX][a-fA-F0-9]+((_[a-fA-F0-9]+)+)?(\.([a-fA-F0-9]+((_[a-fA-F0-9]+)+)?)?)?p[+-]?\d+', Number.Float),
+            (r'0[bB][01]+((_[01]+)+)?', Number.Bin),
+            (r'0[oO][0-7]+((_[0-7]+)+)?', Number.Oct),


Suggested change

(r'0[oO][0-7]+((_[0-7]+)+)?', Number.Oct),

(r'0[oO][0-7]+(_[0-7]+)*', Number.Oct),

jeanas · 2024-03-29T11:56:53Z

pygments/lexers/gleam.py

+            (r'0[xX][a-fA-F0-9]+((_[a-fA-F0-9]+)+)?(\.([a-fA-F0-9]+((_[a-fA-F0-9]+)+)?)?)?p[+-]?\d+', Number.Float),
+            (r'0[bB][01]+((_[01]+)+)?', Number.Bin),
+            (r'0[oO][0-7]+((_[0-7]+)+)?', Number.Oct),
+            (r'0[xX][a-fA-F0-9]+((_[a-fA-F0-9]+)+)?', Number.Hex),


Suggested change

(r'0[xX][a-fA-F0-9]+((_[a-fA-F0-9]+)+)?', Number.Hex),

(r'0[xX][a-fA-F0-9]+(_[a-fA-F0-9]+)*', Number.Hex),

jeanas · 2024-03-29T11:57:04Z

pygments/lexers/gleam.py

+            (r'0[bB][01]+((_[01]+)+)?', Number.Bin),
+            (r'0[oO][0-7]+((_[0-7]+)+)?', Number.Oct),
+            (r'0[xX][a-fA-F0-9]+((_[a-fA-F0-9]+)+)?', Number.Hex),
+            (r'\d+((_\d+)+)?', Number.Integer),


Suggested change

(r'\d+((_\d+)+)?', Number.Integer),

(r'\d+(_\d+)*', Number.Integer),

This and other suggestions have been implemented, thanks for the input

Pi-Cla · 2024-04-12T20:22:28Z

@darenc Hi, are you still working on this?
I have been able to add a lexer to Chroma so I was wondering whether I should let you finish this or if I should try to translate the lexer I made in Chroma into Pygments and make a new PR.

What do you think?

darenc · 2024-04-12T21:22:47Z

@darenc Hi, are you still working on this?

Yep, I'll pick up the requested changes in the next few days. Just been busy with other things.

Anteru · 2024-10-06T15:13:14Z

Merged, thanks!

logan-connolly · 2024-11-28T19:25:38Z

Just wanted to give you guys a heads up that the documentation on your website states that Gleam support was added in 2.18, but it appears here that it is planned for 2.19. Thanks a lot for all your hard work and looking forward to the next release :)

Anteru · 2025-01-06T13:47:28Z

That's fixed as well now, thanks!

darenc and others added 6 commits March 13, 2024 10:55

Initial commit Gleam lexer

66b79f7

Improve Gleam lexing

44bd201

Add large Gleam test file

5cc4c34

Add myself to AUTHORS

952194a

Remove redundant whitespace check

60d719e

Fix CRLFs

f9fb01d

darenc added 2 commits March 14, 2024 07:40

Remove redundant regex flag

2a68da8

Merge branch 'gleam-lexer' of https://github.com/darenc/pygments into…

bf6785a

… gleam-lexer

Anteru requested changes Mar 24, 2024

View reviewed changes

pygments/lexers/gleam.py Outdated Show resolved Hide resolved

pygments/lexers/gleam.py Outdated Show resolved Hide resolved

darenc added 2 commits March 24, 2024 10:54

Update version

2b774e6

Match multiple punctuation characters

3a76f03

darenc requested a review from Anteru March 25, 2024 13:17

jeanas requested changes Mar 29, 2024

View reviewed changes

darenc added 2 commits April 20, 2024 14:37

Merge branch 'pygments:master' into gleam-lexer

6770f48

Update regexes

396dbf8

Anteru merged commit eaa22a2 into pygments:master Oct 6, 2024

Anteru added this to the 2.19.0 milestone Oct 6, 2024

Anteru added the A-lexing area: changes to individual lexers label Oct 6, 2024

	(r'(\d+((_\d+)+)?\.(?!\.)(\d+((_\d+)+)?)?\|\.\d+((_\d+)+)?)([eEf][+-]?[0-9]+)?', Number.Float),
	(r'(\d+(_\d+)\.(?!\.)(\d+(_\d+))?\|\.\d+(_\d+)*)([eEf][+-]?[0-9]+)?', Number.Float),

	(r'\d+((_\d+)+)?[eEf][+-]?[0-9]+', Number.Float),
	(r'\d+(_\d+)*[eEf][+-]?[0-9]+', Number.Float),

	(r'0[xX][a-fA-F0-9]+((_[a-fA-F0-9]+)+)?(\.([a-fA-F0-9]+((_[a-fA-F0-9]+)+)?)?)?p[+-]?\d+', Number.Float),
	(r'0[xX][a-fA-F0-9]+(_[a-fA-F0-9]+)(\.([a-fA-F0-9]+(_[a-fA-F0-9]+))?)?p[+-]?\d+', Number.Float),

	(r'0[bB][01]+((_[01]+)+)?', Number.Bin),
	(r'0[bB][01]+(_[01]+)*', Number.Bin),

	(r'0[oO][0-7]+((_[0-7]+)+)?', Number.Oct),
	(r'0[oO][0-7]+(_[0-7]+)*', Number.Oct),

	(r'0[xX][a-fA-F0-9]+((_[a-fA-F0-9]+)+)?', Number.Hex),
	(r'0[xX][a-fA-F0-9]+(_[a-fA-F0-9]+)*', Number.Hex),

	(r'\d+((_\d+)+)?', Number.Integer),
	(r'\d+(_\d+)*', Number.Integer),

Conversation

darenc commented Mar 13, 2024

Uh oh!

jeanas commented Mar 13, 2024

Uh oh!

darenc commented Mar 13, 2024

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Pi-Cla commented Apr 12, 2024

Uh oh!

darenc commented Apr 12, 2024

Uh oh!

Anteru commented Oct 6, 2024

Uh oh!

logan-connolly commented Nov 28, 2024

Uh oh!

Anteru commented Jan 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants