[UnitTests] Add initial set of dedicated early-exit unit tests. by fhahn · Pull Request #264 · llvm/llvm-test-suite

fhahn · 2025-06-25T11:10:57Z

Adds initial unit tests for early-exit vectorization covering a variation of auto-vectorization and forced interleaving with pragmas.

The interleaving variant is currently mis-compiled and needs

We should probably extend the tests to make sure we cover various other scenarios, including returning the loaded element for the early exit, different index types and array sizes.

david-arm · 2025-06-26T08:56:12Z

@@ -1,3 +1,12 @@
+
+# Enable matrix types extension tests for compilers supporting -fenable-matrix.


I think the comment doesn't match the code below?

Removed now, givne that it has been enabled by default for a while now

david-arm · 2025-06-26T08:58:09Z

+  };                                                                           \
+  auto InterleavedFn = [](std::function<void(int *)> II) -> unsigned {         \
+    Init II(Src);                                                              \
+    _Pragma("clang loop vectorize(enable) interleave_count(4)") Loop           \


Looks like this PR cannot be merged until the pragma has handled correctly.

This should also be working now as expected

Great! Is there a way to tell that these loops did actually vectorise? It's pretty hard to encourage loop vectorisation for early exit loops now with the dereferenceable SCEV checking code having to ensure that the pointers never wrap. I wouldn't be surprised if additional builtins are required to provide attributes to the pointers.

Unfortuantely I don't think there's a way to test if they are actually vectorized expect checking the generated code or remarks.

the current version uses a local array on the stack where the size is directly related to the bounds, which should be the most reliable way we have currently. We could also add ones using the dereferenceable builtins as follow-ups and possible ones without static info to make sure we also have coverage once first-faulting loads are supported

david-arm · 2025-06-26T09:12:30Z

+/// * one where VF is picked automatically and interleaving is forced.
+#define DEFINE_SCALAR_AND_VECTOR_EARLY_EXIT(Init, Src, Loop)                   \
+  auto ScalarFn = [](std::function<void(int *)> II) -> unsigned {              \
+    Init II(Src);                                                              \


This is pretty confusing as it looks like constructing a variable. Can you split this out over different lines so it's clearer, i.e.

InitSrc; II(Src);

updated thanks

david-arm · 2025-06-26T09:15:32Z

+    // for to zero.
+    auto Init1 = [IdxToFind](int *Arr) {
+      std::fill_n(Arr, N, 1);
+      if (IdxToFind < N)


Is there a risk that a future compiler optimisation basically figures out the answer based on this init code (after inlining) and just optimises ScalarFn to return a constant before we even reach the loop vectoriser? It might be better to have Init1 and Init2 as real functions marked as noinline.

In other places we prevent this fro caling through a function marked as optnone. Done the same thing here, thanks

fhahn · 2026-01-12T12:17:12Z

ping

fhahn · 2026-01-19T13:07:18Z

ping

Meinersbur

LGTM

david-arm · 2026-01-19T14:01:37Z

+    II(Src);                                                                   \
+    _Pragma("clang loop vectorize(enable)") Loop                               \
+  };                                                                           \
+  auto InterleavedFn = [](std::function<void(int *)> II) -> unsigned {         \


Is it worth having a variant of this where IC=4, VF=1 to test the interleaved/scalar case?

Added, thanks

Thanks for updating! Although I actually meant something like this

_Pragma("clang loop vectorize_width(1) interleave_count(4)") Loop

. I think both this new one and the one below could still vectorise with VF>1 right?

yep, added an interleaved-only variant as well, thanks

david-arm · 2026-01-19T14:04:08Z

+  };                                                                           \
+  auto InterleavedFn = [](std::function<void(int *)> II) -> unsigned {         \
+    Init II(Src);                                                              \
+    _Pragma("clang loop vectorize(enable) interleave_count(4)") Loop           \


Great! Is there a way to tell that these loops did actually vectorise? It's pretty hard to encourage loop vectorisation for early exit loops now with the dereferenceable SCEV checking code having to ensure that the pointers never wrap. I wouldn't be surprised if additional builtins are required to provide attributes to the pointers.

Adds initial unit tests for early-exit vectorization covering a variation of auto-vectorization and forced interleaving with pragmas. The interleaving variant is currently mis-compiled and needs * llvm/llvm-project#145340 * llvm/llvm-project#145394. We should probably extend the tests to make sure we cover various other scenarios, including returning the loaded element for the early exit, different index types and array sizes.

david-arm

LGTM!

luporl · 2026-01-22T19:24:17Z

@fhahn These new unit tests fail on AArch64 with -mcpu=neoverse-v2:

Can you please fix them or revert this PR?

…s. (#264)" This reverts commit 10a0e5a.

…s. (#264)" (#328) This reverts commit 10a0e5a. Fails on AArch64 with -mcpu=neoverse-v2: https://lab.llvm.org/buildbot/#/builders/198/builds/11196 https://lab.llvm.org/buildbot/#/builders/199/builds/8332

fhahn · 2026-01-22T23:49:45Z

@luporl thanks for the heads up, reverted for now, will take a closer look tomorrow morning

Meinersbur · 2026-01-23T09:25:45Z

Also failed in staging: https://lab.llvm.org/staging/#/builders/49/builds/1274

fhahn · 2026-01-29T12:10:04Z

I looked into the neoverse-v2 failure. The only difference between NEON codegen is that we lower llvm.experimental.cttz.elts.i64.v4i1 using SVE instructions, and that is causing the incorrect result. This seems to indicate a lowering issue llvm/llvm-project#178644

fhahn · 2026-01-29T12:20:28Z

Also failed in staging: https://lab.llvm.org/staging/#/builders/49/builds/1274

This failure looks puzzling and more like an issue with the build directory being stale unless I am missing something, as i t fails with

FAIL: test-suite :: SingleSource/UnitTests/Vectorizer/early-exit.test (2324 of 2465)
******************** TEST 'test-suite :: SingleSource/UnitTests/Vectorizer/early-exit.test' FAILED ********************
/srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/tools/timeit-target --timeout 7200 --limit-core 0 --limit-cpu 7200 --limit-file-size 209715200 --limit-rss-size 838860800 --append-exitstatus --redirect-output /srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/SingleSource/UnitTests/Vectorizer/Output/early-exit.test.out --redirect-input /dev/null --chdir /srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/SingleSource/UnitTests/Vectorizer --summary /srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/SingleSource/UnitTests/Vectorizer/Output/early-exit.test.time /srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/SingleSource/UnitTests/Vectorizer/early-exit
/srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/tools/fpcmp-target /srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/SingleSource/UnitTests/Vectorizer/Output/early-exit.test.out /srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/SingleSource/UnitTests/Vectorizer/early-exit.reference_output
+ /srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/tools/fpcmp-target /srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/SingleSource/UnitTests/Vectorizer/Output/early-exit.test.out /srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/SingleSource/UnitTests/Vectorizer/early-exit.reference_output
/srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/tools/fpcmp-target: error: unable to open '/srv/bbworker/polly-x86_64-fdcserver/polly-x86_64-linux-test-suite/build/testsuite.build/SingleSource/UnitTests/Vectorizer/early-exit.reference_output'

Meinersbur · 2026-01-29T14:06:00Z

@fhahn You are right, the buildbot did not clean the test-suite's build dir, meaning that that *.test files, even if the test disappeared from test-suite, were kept in the build-dir. Fixed in llvm/llvm-project@e509974

…ts. (#264)" (#328) This reverts commit be30466. (#264) Reapply the commit without interleaving forced for now, as this exposed on AArch64 SVE lowering bug: llvm/llvm-project#178644

fhahn · 2026-01-29T22:09:55Z

@fhahn You are right, the buildbot do not clean the test-suite's build dir, meaning that that *.test files, even if the test disappeared from test-suite, were kept in the build-dir. Fixed in llvm/llvm-project@e509974

Great thanks!

I re-landed the PR, but with forced interleaving left out for now until llvm/llvm-project#178644 is fixed so we can make progress with #325

fhahn requested review from annamthomas and david-arm June 25, 2025 11:10

fhahn mentioned this pull request Jun 25, 2025

[LV] Enable auto-vectorisation of loops with uncountable exits llvm/llvm-project#133099

Merged

fhahn force-pushed the test-suite-early-exit-vec branch from b349e09 to 28ae6ea Compare June 25, 2025 18:26

david-arm reviewed Jun 26, 2025

View reviewed changes

fhahn force-pushed the test-suite-early-exit-vec branch from 28ae6ea to fa5c6c3 Compare January 7, 2026 10:09

fhahn mentioned this pull request Jan 19, 2026

[UnitTests] Add initial set of tests with multiple early exits. #325

Merged

Meinersbur approved these changes Jan 19, 2026

View reviewed changes

david-arm reviewed Jan 19, 2026

View reviewed changes

fhahn added 2 commits January 19, 2026 15:05

!fixup address comments, thanks

886d544

fhahn force-pushed the test-suite-early-exit-vec branch from fa5c6c3 to e60f8ff Compare January 19, 2026 15:05

!fixup also add variant with just VF forced.

5dbe493

fhahn force-pushed the test-suite-early-exit-vec branch from e60f8ff to 5dbe493 Compare January 19, 2026 15:06

!fixup add interleave-only test as well

0de01b0

david-arm approved these changes Jan 19, 2026

View reviewed changes

fhahn merged commit 10a0e5a into llvm:main Jan 19, 2026
1 check passed

fhahn deleted the test-suite-early-exit-vec branch January 19, 2026 16:22

fhahn added a commit that referenced this pull request Jan 22, 2026

Revert "[UnitTests] Add initial set of dedicated early-exit unit test…

4af1378

…s. (#264)" This reverts commit 10a0e5a.

fhahn mentioned this pull request Jan 22, 2026

Revert "[UnitTests] Add initial set of dedicated early-exit unit tests." #328

Merged

		@@ -1,3 +1,12 @@

		# Enable matrix types extension tests for compilers supporting -fenable-matrix.

Conversation

fhahn commented Jun 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fhahn commented Jan 12, 2026

Uh oh!

fhahn commented Jan 19, 2026

Uh oh!

Meinersbur left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

david-arm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

luporl commented Jan 22, 2026

Uh oh!

fhahn commented Jan 22, 2026

Uh oh!

Meinersbur commented Jan 23, 2026

Uh oh!

fhahn commented Jan 29, 2026

Uh oh!

fhahn commented Jan 29, 2026

Uh oh!

Meinersbur commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fhahn commented Jan 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Meinersbur commented Jan 29, 2026 •

edited

Loading