fix(lib): address strict aliasing violations with `Array` type by WillLillis · Pull Request #5242 · tree-sitter/tree-sitter

WillLillis · 2026-01-17T07:02:57Z

Altering the Array type itself isn't feasible, as this causes unacceptable breakage with existing parsers that depend on it. Instead, pass in individual Array fields for to various _array__* functions.

Any time the contents of an array may be modified (freed, reallocd, etc), return the potentially new address out by value. This prevents any strict aliasing violations as we're no longer writing to a type-casted pointer.

Note that the array.h included in a grammar's src/tree-sitter/ directory is updated upon running tree-sitter generate.

This is a rework of #5194, thanks @nwf for taking an initial pass and discussing solutions!

Closes "tree-sitter test" crashes on ppc64el since v0.25.7 #4960
Supersedes Address strict aliasing violations in array.h #5194

WillLillis · 2026-01-18T06:08:13Z

I believe the CI failures are now due to this line in tree-sitter-php's scanner:

https://github.com/tree-sitter/tree-sitter-php/blob/master/common/scanner.h#L468

(Here's the snippet since GH won't render it inline)

array_delete(&array_pop(&scanner->heredocs).word);

Taking the address of a non-lvalue is undefined behavior, and this seems to be it. I've opened a PR to fix the issue here (tree-sitter/tree-sitter-php#293), so once that's merged and a new release is published, we can update the tag in test/fixtures/fixtures.json, and then (barring any other issues with other corpus grammars) this PR can proceed.

clason · 2026-01-19T10:10:30Z

Once the php scanner is fixed, I'd like to run a crater test with Neovim built against this PR. (I'm sure that's not the only parser with... questionable behavior out there...)

Set tree-sitter-php fixture to `upstream_test_fixture` branch. Also remove the `--update` flag, as it does nothing.

Altering the `Array` type itself isn't feasible, as this causes unacceptable breakage with existing parsers that depend on it. Instead, pass in individual `Array` fields for to various `_array__*` functions. Any time the `contents` of an array may be modified (`free`d, `realloc`d, etc), return the potentially new address out by value. This prevents any strict aliasing violations as we're no longer writing to a type-casted pointer. Co-authored-by: Nathaniel Wesley Filardo <nwfilardo@gmail.com>

WillLillis · 2026-01-21T05:15:58Z

I've added 29281b1, which allows fixtures.json to specify an override branch for a test fixture grammar to be checked out to. I've created the upstream_test_fixture branch in the tree-sitter-php repo, and have pointed the fixtures.json file to use that branch instead of the v0.24.2 tag. The changes in this branch include fixing the undefined behavior described in this comment, as well as regenerating the parser in order to update the repo's array.h copies.

CC @clason I think this is a better solution than disabling the PHP tests, especially given how hard to predict/reason about some of these crashes are.

nwf · 2026-01-21T05:23:45Z

I am not entirely sure that this will close #4960, though it is surely a huge improvement, since, as I noted over in #4960 (comment), there is also some strict-aliasing violation going on in tree_cursor.c. I wouldn't be surprised if that isn't biting anyone yet, but, strictly speaking...

WillLillis · 2026-01-21T05:51:37Z

I am not entirely sure that this will close #4960, though it is surely a huge improvement, since, as I noted over in #4960 (comment), there is also some strict-aliasing violation going on in tree_cursor.c. I wouldn't be surprised if that isn't biting anyone yet, but, strictly speaking...

I was planning on extracting the violations in tree_cursor.c to a separate issue and keep #4960 scoped to the crashes observed on ppc64el (I still need to fight with docker in order to test these changes on there again.). I'm planning on taking a look at tree_cursor.c after we get this resolved. 🙂

nwf · 2026-01-21T14:04:01Z

Sounds like a plan! Thank you!

WillLillis · 2026-01-22T03:55:53Z

I've gone through most of the grammars with external scanners in nvim-treesitter to check for further fallout from this change. Besides tree-sitter-php, The following repos crashed when regenerated and tested:

clason · 2026-01-24T18:08:13Z

also

tree-sitter-vue: renovate tree-sitter-grammars/tree-sitter-vue#11

All three PRs are now merged, so this is good to go?

tree-sitter-ci-bot · 2026-01-24T19:18:54Z

Successfully created backport PR for release-0.26:

fix(lib): address strict aliasing violations with Array type #5254

nwf · 2026-01-24T20:16:41Z

Belatedly (sorry!), can say that cargo xtask test looks to have only unrelated failures on my powerpc64le Debian box. (Not quite sure why "tests::corpus_test::test_feature_corpus_files" is failing with "called Result::unwrap() on an Err value: ScannerSymbols(ScannerSymbolError { missing: ["tree_sitter_external_unicode_column_alignment_external_scanner_create", ...", but it is, but it doesn't seem relevant).

WillLillis · 2026-01-24T21:29:26Z

Belatedly (sorry!), can say that cargo xtask test looks to have only unrelated failures on my powerpc64le Debian box. (Not quite sure why "tests::corpus_test::test_feature_corpus_files" is failing with "called Result::unwrap() on an Err value: ScannerSymbols(ScannerSymbolError { missing: ["tree_sitter_external_unicode_column_alignment_external_scanner_create", ...", but it is, but it doesn't seem relevant).

That shouldn't be relevant to these changes, but I'm not sure why you'd get that error. Have you cargo xtask fetch-fixtures and cargo xtask generate-fixtures?

WillLillis · 2026-01-28T06:41:34Z

Following up on this, the fixture corpus tests are now passing for me in a ppcle64 ubuntu docker container. I have ran into the same missing symbol issue raised by @nwf above. The problem seems to be here:

tree-sitter/crates/loader/src/loader.rs

Lines 1298 to 1302 in 4f8983b

    
           let section = if cfg!(all(target_arch = "powerpc64", target_os = "linux")) { 
        
               " D " 
        
           } else { 
        
               " T " 
        
           };

For some background, on non-Windows platforms we use the system installed nm to verify symbols in the generated parser shared library. There is a special case for powerpc64 to use the D section of nm's output, and T for all other architectures. It looks like with the a more recent version of binutils, ld now places the functions in the correct section, or something in nm changed to indicate the correct section. Potentially related discussion:

https://lists.gnu.org/archive/html/bug-binutils/2021-11/msg00024.html

WillLillis marked this pull request as draft January 17, 2026 07:36

nwf mentioned this pull request Jan 17, 2026

Address strict aliasing violations in array.h #5194

Closed

WillLillis force-pushed the array_aliasing branch from 40d79d3 to e9b1de7 Compare January 18, 2026 03:50

WillLillis mentioned this pull request Jan 18, 2026

fix(scanner): correct UB in scan's heredoc delete logic tree-sitter/tree-sitter-php#293

Merged

WillLillis marked this pull request as ready for review January 18, 2026 06:08

WillLillis added c-library ci:backport release-0.26 Backport label labels Jan 18, 2026

WillLillis force-pushed the array_aliasing branch from e9b1de7 to f452d38 Compare January 21, 2026 04:52

WillLillis and others added 2 commits January 20, 2026 23:53

feat(xtask): allow alternate branch for fixture grammars

29281b1

Set tree-sitter-php fixture to `upstream_test_fixture` branch. Also remove the `--update` flag, as it does nothing.

WillLillis force-pushed the array_aliasing branch from f452d38 to 68ec705 Compare January 21, 2026 04:53

WillLillis mentioned this pull request Jan 21, 2026

fix(scanner): correct UB when deleting old tag Strings tree-sitter-grammars/tree-sitter-xml#36

Merged

WillLillis mentioned this pull request Jan 22, 2026

fix(scanner): correct UB in nested_scan tlaplus-community/tree-sitter-tlaplus#142

Merged

clason approved these changes Jan 24, 2026

View reviewed changes

WillLillis merged commit 5177b3d into tree-sitter:master Jan 24, 2026
20 checks passed

WillLillis deleted the array_aliasing branch January 24, 2026 19:18

tree-sitter-ci-bot bot mentioned this pull request Jan 24, 2026

fix(lib): address strict aliasing violations with Array type #5254

Merged

WillLillis mentioned this pull request Jan 28, 2026

fix(loader): account for nm/ld fix on newer powerpc linux toolchains #5268

Merged

amaanq mentioned this pull request Feb 8, 2026

lib: clean up strict aliasing fixes in array.h #5309

Merged

Uh oh!

Conversation

WillLillis commented Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WillLillis commented Jan 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clason commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WillLillis commented Jan 21, 2026

Uh oh!

nwf commented Jan 21, 2026

Uh oh!

WillLillis commented Jan 21, 2026

Uh oh!

nwf commented Jan 21, 2026

Uh oh!

WillLillis commented Jan 22, 2026

Uh oh!

clason commented Jan 24, 2026

Uh oh!

Uh oh!

tree-sitter-ci-bot bot commented Jan 24, 2026

Uh oh!

nwf commented Jan 24, 2026

Uh oh!

WillLillis commented Jan 24, 2026

Uh oh!

WillLillis commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

WillLillis commented Jan 17, 2026 •

edited

Loading

WillLillis commented Jan 18, 2026 •

edited

Loading

clason commented Jan 19, 2026 •

edited

Loading

WillLillis commented Jan 28, 2026 •

edited

Loading