Add support for \expandafter, \noexpand, \edef, \let, and \long by ylemkimon · Pull Request #2122 · KaTeX/KaTeX

ylemkimon · 2019-10-12T15:18:24Z

I'm surprised that they are fairly simple to implement! Fixes #1413.

codecov-io · 2019-10-12T15:20:20Z

Codecov Report

Merging #2122 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2122   +/-   ##
=======================================
  Coverage   94.77%   94.77%           
=======================================
  Files          83       83           
  Lines        5358     5358           
  Branches      939      939           
=======================================
  Hits         5078     5078           
  Misses        257      257           
  Partials       23       23

Flag	Coverage Δ
#screenshotter	`88.45% <ø> (ø)`	⬆️
#test	`88.55% <ø> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7eb2815...c0307bc. Read the comment docs.

ylemkimon · 2019-10-19T11:50:24Z

This is ready for review.

k4b7 · 2019-12-02T00:34:39Z

Since this depends on #2138, maybe set the base of this PR to be internal instead of `master.

k4b7

Thanks for the PR. I'm familiar with these commands so most of my comments are questions. 😅 It would be good to have someone with more LaTeX knowledge review if possible. I do think that introducing some named constants for the different noexpand values would improve readability.

k4b7 · 2020-02-21T22:52:42Z

docs/supported.md

-`\gdef` and `\global\def` macros will persist between math expressions.
+`\gdef`, `\xdef`, `\global\def`, `\global\edef`, `\global\let`, and `\global\futurelet` will persist between math expressions.
+
+KaTeX has no `\par`, so all macros are long by default and `\long` will be ignored.


TIL about \long and \par.

k4b7 · 2020-02-21T23:01:37Z

src/Token.js

 export class Token {
    text: string;
    loc: ?SourceLocation;
+    noexpand: ?number; /* 1: treat as \relax, 2: pass to the parser */


Having constants for these would make the code clearer.

k4b7 · 2020-02-21T23:06:21Z

src/functions/def.js

+// <assignment> -> <non-macro assignment>|<macro assignment>
+// <non-macro assignment> -> <simple assignment>|\global<non-macro assignment>
+// <macro assignment> -> <definition>|<prefix><macro assignment>
+// <prefix> -> \global|\long|\outer


What does this mean?

It's the (formal) grammar of TeX from the TeXbook

k4b7 · 2020-02-21T23:11:32Z

src/functions/def.js

+    names: [
+        "\\global", "\\long",
+        "\\\\globallong", // can’t be entered directly
+    ],


Why are \global and \long grouped together?

They work in the same manner as a prefix to the macro assignment.

k4b7 · 2020-02-21T23:13:58Z

src/functions/def.js

+    // if macro is undefined at this moment, set noexpand to 2
+    // and unexpandable to not expand it later and pass to the parser


Is this to handle a case like \let\foo=\bar \def\bar{hello, world} \foo where the value of the let is defined after the let?

This is to handle cases where non-macros are overwritten after \let, e.g., \let\foo=\frac \def\frac{123} \foo12.

k4b7 · 2020-02-21T23:42:38Z

test/katex-spec.js

+        expect`\noexpand\foo y`.toParseLike("y",
+            new Settings({macros: {"\\foo": "x"}}));


It looks like the \foo is ignored in this case. Is that correct?

k4b7 · 2020-02-21T23:45:07Z

test/katex-spec.js

+        expect`\def\foo{a}\xdef\bar{\foo}\def\foo{}\bar`.toParseLike`a`;
+        expect`\def\foo{a}\xdef\bar{\def\noexpand\foo{}}\foo\bar\foo`.toParseLike`a`;
+        expect`\def\foo{a}\xdef\bar{\expandafter\foo\noexpand\foo}\def\foo{b}\bar`
+            .toParseLike`ab`;


I get the first one, but the second two are a bit mind warping.

k4b7 · 2020-02-21T23:46:54Z

test/katex-spec.js

+        expect`\global\global\def\foo{}\foo`.toParseLike``;
+        expect`\global\long\def\foo{}\foo`.toParseLike``;


It looks like multiple global/long commands in a row are the same as having a single one of those commands, is that right?

k4b7 · 2020-02-21T23:52:31Z

test/katex-spec.js

+        expect`\let\foo=\frac\def\frac{}\foo12`.toParseLike`\frac12`;
+        expect`\def\foo{1}\let\bar\foo\def\foo{2}\bar`.toParseLike`1`;
+        expect`\let\foo=\kern\edef\bar{\foo1em}\let\kern=\relax\bar`.toParseLike`\kern1em`;
+        expect`\let\foo{\frac\foo1}{2}`.toParseLike`\frac{1}{2}`;


I didn't realize the = was optional when using \let. In the first one it looks like \frac is being redefined, but the result doesn't include this redefinition. I think it would be useful to have comments for some of these tests to help people that aren't as familiar with the interections between let and def.

I'll add comments on test cases.

k4b7 · 2020-02-21T23:55:18Z

test/__snapshots__/katex-spec.js.snap

+                  "input": "\\\\def\\\\bold{\\\\bgroup\\\\bf\\\\let\\\\next= }\\\\bold{a}",
+                  "lastIndex": 40


It's nice seeing the input here. It makes reviewing snapshot tests easier.

k4b7

LGTM. The comments were very helpful. Thank you!

k4b7 · 2020-03-08T00:22:31Z

src/Token.js

+    noexpand: ?boolean; // don't expand the token
+    treatAsRelax: ?boolean; // used in \noexpand


Is any combination of these valid?

It's possible in theory, but there is no use yet.

ylemkimon added 3 commits October 13, 2019 02:12

Add support for \expandafter

5445ef8

Add support for \noexpand

3658305

Add support for \edef

5be7b5a

ylemkimon force-pushed the expandafter branch from df69352 to 5be7b5a Compare October 12, 2019 17:30

ylemkimon changed the title ~~Add support for \expandafter~~ Add support for \expandafter, \noexpand, and \edef Oct 12, 2019

ylemkimon added 2 commits October 13, 2019 02:33

Update comments

b91c5cb

Allow \long before macro definition

289d75a

ylemkimon changed the title ~~Add support for \expandafter, \noexpand, and \edef~~ Add support for \expandafter, \noexpand, \edef, and \long Oct 12, 2019

ylemkimon added 5 commits October 13, 2019 02:57

Update documentation

1e37149

Merge branch 'master' into expandafter

8547c35

Update comments

ffdb490

Fix defPrefix

826a7dc

Add support for \let

da1a95c

ylemkimon changed the title ~~Add support for \expandafter, \noexpand, \edef, and \long~~ Add support for \expandafter, \noexpand, \edef, \let, and \long Oct 13, 2019

ylemkimon added 3 commits October 14, 2019 00:26

Update documentation

f493701

Print error token

7378f50

Update documentation

2d15b29

ylemkimon changed the title ~~Add support for \expandafter, \noexpand, \edef, \let, and \long~~ [WIP]Add support for \expandafter, \noexpand, \edef, \let, and \long Oct 14, 2019

ylemkimon added 4 commits October 19, 2019 17:22

Check whether command is expandable

ce15ecb

Add tests

c47942a

Fix token order

186471e

Make noexpand a Token property

22b46d0

ylemkimon changed the title ~~[WIP]Add support for \expandafter, \noexpand, \edef, \let, and \long~~ Add support for \expandafter, \noexpand, \edef, \let, and \long Oct 19, 2019

ylemkimon added 5 commits October 20, 2019 00:27

Throw error if control sequence is undefined when expanding

7f46df2

Rename expandableOnly to expandOnly

93ead3e

Make unexpandable macro property

cb08a30

Move \expandafter to macros.js

f9405a8

Add TODO

495df93

ylemkimon mentioned this pull request Dec 1, 2019

Move \global and \def to functions #2138

Merged

ylemkimon added 3 commits December 7, 2019 13:29

Merge branch 'master' into expandafter

fbf388e

Fix merge conflict

9ca8fad

Update a test case

f61a86f

ylemkimon added the GH Review: review-needed label Dec 7, 2019

ylemkimon added 4 commits December 7, 2019 14:12

Remove unused functions in MacroContextInterface

8b31bbc

Update comments

bc2fe40

Refactor code

ec1feff

Move \noexpand to macros

638ead3

ylemkimon force-pushed the expandafter branch from a7416e8 to 638ead3 Compare December 7, 2019 15:56

ylemkimon and others added 8 commits December 8, 2019 01:18

Update MacroExpander.js

0af94b9

Add a test case

0b12709

Separate control sequence check to a function

d5e94e9

Add support for \futurelet

eabef1a

Separate RHS getter to a function

0889814

Update documentation

34c7f42

Move expandOnly logic to expandOnce

721d966

Merge branch 'master' into expandafter

89c679f

k4b7 reviewed Feb 21, 2020

View reviewed changes

ylemkimon added 2 commits February 24, 2020 05:08

Refactor code and update comments

7eb2815

Merge branch 'master' into expandafter

c0307bc

k4b7 approved these changes Mar 8, 2020

View reviewed changes

ylemkimon merged commit 9917d1c into master Mar 11, 2020

ylemkimon deleted the expandafter branch July 11, 2020 19:19

		// if macro is undefined at this moment, set noexpand to 2
		// and unexpandable to not expand it later and pass to the parser

		expect`\noexpand\foo y`.toParseLike("y",
		new Settings({macros: {"\\foo": "x"}}));

		expect`\global\global\def\foo{}\foo`.toParseLike``;
		expect`\global\long\def\foo{}\foo`.toParseLike``;

		"input": "\\\\def\\\\bold{\\\\bgroup\\\\bf\\\\let\\\\next= }\\\\bold{a}",
		"lastIndex": 40

		noexpand: ?boolean; // don't expand the token
		treatAsRelax: ?boolean; // used in \noexpand

Uh oh!

Conversation

ylemkimon commented Oct 12, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-io commented Oct 12, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ylemkimon commented Oct 19, 2019

Uh oh!

k4b7 commented Dec 2, 2019

Uh oh!

k4b7 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ylemkimon Feb 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ylemkimon Feb 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

k4b7 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ylemkimon commented Oct 12, 2019 •

edited

Loading

codecov-io commented Oct 12, 2019 •

edited

Loading

ylemkimon Feb 23, 2020 •

edited

Loading

ylemkimon Feb 23, 2020 •

edited

Loading