Skip to content

feat: Add support for array repeat syntax: [<element_expr>; <cnt_expr>]#626

Merged
GuillaumeGomez merged 1 commit intoaskama-rs:mainfrom
seijikun:mr-array-repeat
Nov 25, 2025
Merged

feat: Add support for array repeat syntax: [<element_expr>; <cnt_expr>]#626
GuillaumeGomez merged 1 commit intoaskama-rs:mainfrom
seijikun:mr-array-repeat

Conversation

@seijikun
Copy link
Copy Markdown
Contributor

@seijikun seijikun commented Nov 24, 2025

Please take a good look at the parser stuff and make sure I didn't accidentally break normal arrays 😅

Implements: #625

@GuillaumeGomez
Copy link
Copy Markdown
Collaborator

Implementation looks good to me, thanks! Can you add a GUI test for cases like [0;] please?

@GuillaumeGomez
Copy link
Copy Markdown
Collaborator

Also need to check what's wrong with CI.

@GuillaumeGomez
Copy link
Copy Markdown
Collaborator

Fixed CI in #627. Can you rebase on main so CI can run on your code please?

@seijikun
Copy link
Copy Markdown
Contributor Author

The fuzzer seems unhappy. Did I break something?

@GuillaumeGomez
Copy link
Copy Markdown
Collaborator

You introduced an infinite loop apparently from what I can see.

Comment thread testing/tests/ui/invalid_array_repeat.rs
Comment thread testing/tests/ui/invalid_array_repeat.stderr Outdated
@GuillaumeGomez
Copy link
Copy Markdown
Collaborator

Not similar but for example in 5e5d576 I improved errors from #602. So you need to do a bit more "by hand". Like if you have [ char, then it means that if the parser fails from this point, it means the user wrote some wrong array code and it should be reported.

@seijikun
Copy link
Copy Markdown
Contributor Author

Something like this?

@GuillaumeGomez
Copy link
Copy Markdown
Collaborator

Much better! Although spans are not there yet. :)

Comment thread askama_parser/src/expr.rs
@GuillaumeGomez
Copy link
Copy Markdown
Collaborator

Ok so errors looks good, well done! Now about the regression introduced in this PR, you can reproduce it with https://github.com/askama-rs/askama/actions/runs/19646142627/artifacts/4664834571. Please also add it as a testcase to prevent future regressions.

@seijikun
Copy link
Copy Markdown
Contributor Author

seijikun commented Nov 25, 2025

So, if I understood it correctly, the fuzzing test shows that this:

use askama::Template;

#[derive(Template)]
#[template(
    source = r#"*{{d~[d~[d~[d~[d~[dd~[d~[d~[d~[d~[d~[D~[d~[d~[d~[d~[d~[D~[d~[dd~[d~[d~[d~[d~[d~[D~[d~[d~[d~[d~[d~[d~["#,
    ext = "txt"
)]
struct FuzzingMR626;

fn main() {}

should crash the parser while compiling. But it works locally for me. I just get a proper compile error:

error: expected element expression for array repeat syntax
 --> <source attribute>:1:101
       ""
 --> tests/ui/fuzzed_array_repeat.rs:5:14
  |
5 |     source = r#"*{{d~[d~[d~[d~[d~[dd~[d~[d~[d~[d~[d~[D~[d~[d~[d~[d~[d~[D~[d~[dd~[d~[d~[d~[d~[d~[D~[d~[d~[d~[d~[d~[d~["#,
  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

@seijikun
Copy link
Copy Markdown
Contributor Author

Now that it's added as ui-test, it also succeeds in the CI 🤔
If I understand it correctly, it's an execution timeout hitting:

ALARM: working on the last Unit for 26 seconds
       and the timeout value is 25 (use -timeout=N to change)
MS: 1 InsertByte-; base unit: 277979817b963d163b1e362606013030a6416b77
0x2a,0x7b,0x7b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x44,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x44,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x44,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0xeb,0x7e,0x5b,0x64,0x7e,0x5b,0x64,0x7e,0x5b,
*{{d~[d~[d~[d~[d~[dd~[d~[d~[d~[d~[d~[D~[d~[d~[d~[d~[d~[D~[d~[dd~[d~[d~[d~[d~[d~[D~[d~[d~[d~[d\353~[d~[d~[
artifact_prefix='/tmp/tmpyszgve9c/'; Test unit written to /tmp/tmpyszgve9c/timeout-5b3d17454dfbee1fdbdeedce6b6dd76834468e29
Base64: Knt7ZH5bZH5bZH5bZH5bZH5bZGR+W2R+W2R+W2R+W2R+W2R+W0R+W2R+W2R+W2R+W2R+W2R+W0R+W2R+W2RkfltkfltkfltkfltkfltkfltEfltkfltkfltkfltk635bZH5bZH5b
==26== ERROR: libFuzzer: timeout after 26 seconds

Is there something I need/can change in my implementation to fix this?

@GuillaumeGomez
Copy link
Copy Markdown
Collaborator

I think adding a limit on recursion (like 10 levels?) is the only solution.

@seijikun
Copy link
Copy Markdown
Contributor Author

But isn't this something that can generally produce problems?
Like with function calls:

    {{ test(test(test(test(test(test(test(test(test(test(test(test(test(test()))))))))))))) }}

How come it only occured with this array repeat implementation now?
Shouldn't the previous implementation for normal arrays have the same problem?

@GuillaumeGomez
Copy link
Copy Markdown
Collaborator

We have a nesting protection with InputStream.state.level which we use like this:

// Level is decreased when `_level_guard` is dropped.
let _level_guard = i.state.level.nest(i)?;

Can you add this line in the new array parsing please?

@seijikun seijikun force-pushed the mr-array-repeat branch 2 times, most recently from f921b10 to 1a2c2bc Compare November 25, 2025 12:52
@seijikun
Copy link
Copy Markdown
Contributor Author

seijikun commented Nov 25, 2025

Ahh, there was already a mechanism for this. I had no idea how to add a recursion check on this 😅

I refactored the implementation a little. Now, the backtracking scope is cut off after finding [ (like in the previous array implementation), and from there on - it can be either a normal value array, or the repeat syntax.
Please have another look.
Hope is, that this is more performant, because the backtracking scope is reduced.

@GuillaumeGomez
Copy link
Copy Markdown
Collaborator

This is great, thanks! Please just add one more test like [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[ (you got the idea) to ensure we have the nested error triggered as well. After that the PR looks ready to me. :)

@GuillaumeGomez GuillaumeGomez changed the title feat: Add support for array repeat syntax: [<element_expr>; <cnt_expr>] feat: Add support for array repeat syntax: [<element_expr>; <cnt_expr>] Nov 25, 2025
@GuillaumeGomez
Copy link
Copy Markdown
Collaborator

Thanks a lot!

@GuillaumeGomez GuillaumeGomez merged commit 1427493 into askama-rs:main Nov 25, 2025
51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants