Grammar Syntax#

# comment ...#

Python-style end-of-line comments are allowed.

// comment ...#

Java-style end-of-line comments are allowed.

/* ... */#

EBNF-style multi-line comments are allowed.

e1 | e2#

Choice. Match either e1 or e2.

A | may be used before the first option if desired:

choices:
  | e1
  | e2
  | e3

e1 e2#

Sequence. Match e1 and then match e2.

( e )#

Grouping. Match e. For example: ('a' | 'b').

(?: e )#

Skip. Like, a group, match e, but do not capture what was pared. For example: (?: delimiters ).

[ e ]#

Optionally match e.

{ e } or { e }*#

Closure. Match e zero or more times. The AST returned for a closure is always a list.

{ e }+#

Positive closure. Match e one or more times. The AST is always a list.

{}#

Empty closure. Match nothing and produce an empty list as AST.

~#

The cut expression: commit to the current option and prevent other options from being considered even when what follows fails to parse.

In this example, other options won’t be considered if a parenthesis is parsed:

  atom:
      | '(' ~ @:expre ')'
      | int
      | bool

Cut expression may be used anywhere. The effect of ~ is scoped to the nearest enclosing brackets (group, optional, closure), the enclosing choice, or the enclosing rule.

On the scoping of cut, consider these theoretical equivalences about implicit choices in some expressions:

A →  α   ≡ A → α | ⊥
A → [x]  ≡ A → B, B → x | ε
A → {x}  ≡ A → B, B → xB | ε
A → {x}+ ≡ A → B, B → xB | x

This is a common use of ~. The closure doesn’t allow a partial assignment expressions to escape it:

parameters: ','.{name '=' ~ expression}

s%{ e }+#

Positive join. Inspired by Python’s str.join(), it parses the same as this expression:

e {s ~ e}

yet the result is a single list of the form:

[e, s, e, s, e, ...]

Use grouping if s is more complex than a token or a pattern:

(s t)%{ e }+

s%{ e } or s%{ e }*#

Join. Parses the list of s-separated expressions, or the empty closure. It’s equivalent to:

s%{e}+|{}

op<{ e }+#

Left join. Like the join expression, but the result is a left-associative tree built with tuple(), in which the first element is the separator (op), and the other two elements are the operands.

The expression:

'+'<{/\d+/}+

Will parse this input:

1 + 2 + 3 + 4

To this tree:

(
    '+',
    (
        '+',
        (
            '+',
            '1',
            '2'
        ),
        '3'
    ),
    '4'
)

op>{ e }+#

Right join. Like the join expression, but the result is a right-associative tree built with tuple(), in which the first element is the separator (op), and the other two elements are the operands.

The expression:

'+'>{/\d+/}+

Will parse this input:

1 + 2 + 3 + 4

To this tree:

(
   '+',
   '1',
   (
       '+',
       '2',
       (
           '+',
           '3',
           '4'
       )
   )
)

s.{ e }+#

Positive gather. Like positive join, but the separator is not included in the resulting AST.

s.{ e } or s.{ e }*#

Gather. Like the join, but the separator is not included in the resulting AST. It’s equivalent to:

s.{e}+|{}

&e#

Positive lookahead. Succeed if e can be parsed, but do not consume any input.

!e#

Negative lookahead. Fail if e can be parsed, and do not consume any input.

'text' or "text"#

Match the token text within the quotation marks.

Note that if text is alphanumeric, then 竜 TatSu will check that the character following the token is not alphanumeric. This is done to prevent tokens like IN matching when the text ahead is INITIALIZE. This feature can be turned off by passing nameguard=False to the Parser or the Buffer, or by using a pattern expression (see below) instead of a token expression. The @@nameguard and @@namechars directives may be specified in the grammar for the same effect:

@@nameguard :: False

or to specify additional characters that should also be considered part of names:

@@namechars :: '$-.'

r'text' or r"text"#

Match the token text within the triple quotation marks, interpreting text like Python’s raw string literals. This is useful

'''text''' or """text"""#

A multi-line version of token. The text within the triple quotation marks is stripped of trailing whitespace, and the common indentation is removed as to help with pretty formatting grammars without having to worry about indentation. The resulting text is matched as a token.

/regexp/#

Also ?"regexp" or ?'regexp', The pattern expression. Match the Python regular expression regexp at the current text position. Unlike other expressions, this one does not advance over whitespace or comments. For that, place the regexp as the only term in its own rule.

The regex is interpreted as a Python raw string literal and passed the Python re module using match() at the current position in the text. The returned AST has the semantics of re.findall(pattern, text)[0] (a tuple if there is more than one group), so use (?:) for groups that should not be in the resulting AST.

Consecutive patterns are concatenated to form a single one.

/./#

The any expression, matches the next position in the input. It works exactly like the ?'.' pattern, but is implemented at the lexical level, without regular expressions.

->e#

The “skip to” expression, useful for writing recovery rules.

The parser will advance over input, one character at time, until e matches. Whitespace and comments will be skipped at each step. Advancing over input is done efficiently, with no regular expressions involved.

The expression is equivalent to:

{ !e /./ } e

A common form of the expression is ->&e, which is equivalent to:

{ !e /./ } &e

This is an example of the use of the “skip to” expression for recovery:

statement:
    | if_statement
    # ...

if_statement:
    | 'if' condition 'then' statement ['else' statement]
    | 'if' statement_recovery

statement_recovery: ->&statement

`constant`#

Match nothing, but behave as if constant had been parsed.

Constants can be used to inject elements into the concrete and abstract syntax trees, perhaps avoiding having to write a semantic action. For example:

boolean_option: name ['=' (boolean|`true`) ]

If the text evaluates to a Python literal (with ast.literal_eval()), that will be the returned value. Otherwise, string interpolation in the style of str.format() over the names in the current AST is applied for constant elements. Occurrences of the { character must be escaped to \{ if they are not intended for interpolation. A constant expression that hast type str is evaluated using:

eval(f'{"f" + repr(text)}', {}, ast)

```constant```#

A multi-line version of `constant`.

^`constant`#

Also ^```constant```. An alert. There will be no token returned by the parser, but an alert will be registed in the parse context and added to the current node’s parseinfo.

The ^ character may appear more than once to encode the alert level:

assignment: identifier '=' (
    | value
    | -> &';' ^^^`could not parse value in assignment to {identifier}`
)

rulename#

Invoke the rule named rulename. To help with lexical aspects of grammars, rules with names that begin with an uppercase letter will not advance the input over whitespace or comments.

>rulename#

The include operator. Include the right hand side of rule rulename at this point.

The following set of declarations:

includable: exp1

expanded: exp0 >includable exp2

Has the same effect as defining expanded as:

expanded: exp0 exp1 exp2

Note that the included rule must be defined before the rule that includes it.

()#

The empty expression. Succeed without advancing over input. Its value is the empty tuple ().

!()#

The fail expression. This is ! applied to (), which always fails, and thus has no value. This is useful to prevent the parser from accepting certain input, for example in placeholders for rules that are not yet defined.

name=e or name:e#

Add the result of e to the AST using name as key. If name collides with any attribute or method of dict, or is a Python keyword, an underscore (_) will be appended to the name.

When there are no named items in a rule or choice, the AST consists of the elements parsed by the rule, either a single item or a list. This default behavior makes it easier to write simple rules:

number: /[0-9]+/

Without having to write:

number: number=/[0-9]+/

When a rule has named elements, the unnamed ones are excluded from the AST (they are ignored).

name+=e or name+:e#

Add the result of e to the AST using name as key. Force the AST entry to be a list even if only one element is added. Collisions with dict attributes or Python keywords are resolved by appending an underscore to name.

=e or @:e#

The override operator. Make the AST for the complete rule or choice be the AST for e.

This expression is useful to recover only part of the right hand side of a rule without the need to name it, or add a semantic action. This is a typical use of the override operator:

subexp: '(' =expre ')'

The AST returned for the subexp rule will be the AST recovered from invoking expre.

+=e or @+:e#

Like =e, but make the AST always be a list.

This operator is convenient in cases such as this, in which the delimiting tokens are of no interest.

arglist: '(' +=arg {',' +=arg}* ')'

$#

The end of text symbol. Verify that the end of the input text has been reached.

?/regexp/?#

Another form of the pattern expression that can be used when there are slashes ( / ) in the pattern is deprecated. Use the ?"regexp" or ?'regexp' forms instead.

+/regexp/#

Also +?"regexp" or +?'regexp', concatenate the given pattern with the preceding one is deprecated. Use ?" ..." string concatenations instead.

(* comment *)#

Pascal-style multi-line comments are deprecated. Use Java-style comments instead.