[X86] Use unsigned comparison for stack clash probing loop#192355
Merged
nickdesaulniers merged 1 commit intoApr 16, 2026
Conversation
The stack clash probing loop generated in `EmitLoweredProbedAlloca` used a signed comparison (`X86::COND_GE`) to determine when the allocation target had been reached. In 32-bit mode, memory addresses above `0x80000000` have the sign bit set. If the stack pointer lands in this region, treating the addresses as signed integers causes the comparison logic to fail. This leads to incorrect loop execution, resulting in an infinite loop and a crash (segmentation fault) when setting up custom stacks for pthreads mapped above `0x80000000` in a 32b process. This patch changes the condition code to `X86::COND_AE` (Above or Equal), which generates an unsigned comparison. This ensures that addresses are treated correctly as unsigned quantities on all targets. On 64-bit systems, this change has no practical effect on valid user-space addresses because they do not use the sign bit (being restricted to the lower half of the address space). However, using unsigned comparison is the correct behavior for pointer arithmetic and bounds checks. Reported-by: Wonsik Kim <wonsik@google.com>
Member
|
@llvm/pr-subscribers-backend-x86 Author: Nick Desaulniers (nickdesaulniers) ChangesThe stack clash probing loop generated in In 32-bit mode, memory addresses above This patch changes the condition code to On 64-bit systems, this change has no practical effect on valid Reported-by: Wonsik Kim <wonsik@google.com> Full diff: https://github.com/llvm/llvm-project/pull/192355.diff 3 Files Affected:
diff --git a/llvm/lib/Target/X86/X86ISelLowering.cpp b/llvm/lib/Target/X86/X86ISelLowering.cpp
index e647286301d6b..6a00f57e78dd3 100644
--- a/llvm/lib/Target/X86/X86ISelLowering.cpp
+++ b/llvm/lib/Target/X86/X86ISelLowering.cpp
@@ -37069,7 +37069,7 @@ X86TargetLowering::EmitLoweredProbedAlloca(MachineInstr &MI,
BuildMI(testMBB, MIMD, TII->get(X86::JCC_1))
.addMBB(tailMBB)
- .addImm(X86::COND_GE);
+ .addImm(X86::COND_AE);
testMBB->addSuccessor(blockMBB);
testMBB->addSuccessor(tailMBB);
diff --git a/llvm/test/CodeGen/X86/stack-clash-dynamic-alloca.ll b/llvm/test/CodeGen/X86/stack-clash-dynamic-alloca.ll
index b7d1b593d1ab6..b89532e0f17a1 100644
--- a/llvm/test/CodeGen/X86/stack-clash-dynamic-alloca.ll
+++ b/llvm/test/CodeGen/X86/stack-clash-dynamic-alloca.ll
@@ -16,12 +16,12 @@ define i32 @foo(i32 %n) local_unnamed_addr #0 {
; CHECK-X86-64-NEXT: andq $-16, %rcx
; CHECK-X86-64-NEXT: subq %rcx, %rax
; CHECK-X86-64-NEXT: cmpq %rsp, %rax
-; CHECK-X86-64-NEXT: jge .LBB0_3
+; CHECK-X86-64-NEXT: jae .LBB0_3
; CHECK-X86-64-NEXT: .LBB0_2: # =>This Inner Loop Header: Depth=1
; CHECK-X86-64-NEXT: xorq $0, (%rsp)
; CHECK-X86-64-NEXT: subq $4096, %rsp # imm = 0x1000
; CHECK-X86-64-NEXT: cmpq %rsp, %rax
-; CHECK-X86-64-NEXT: jl .LBB0_2
+; CHECK-X86-64-NEXT: jb .LBB0_2
; CHECK-X86-64-NEXT: .LBB0_3:
; CHECK-X86-64-NEXT: movq %rax, %rsp
; CHECK-X86-64-NEXT: movl $1, 4792(%rax)
@@ -45,12 +45,12 @@ define i32 @foo(i32 %n) local_unnamed_addr #0 {
; CHECK-X86-32-NEXT: andl $-16, %ecx
; CHECK-X86-32-NEXT: subl %ecx, %eax
; CHECK-X86-32-NEXT: cmpl %esp, %eax
-; CHECK-X86-32-NEXT: jge .LBB0_3
+; CHECK-X86-32-NEXT: jae .LBB0_3
; CHECK-X86-32-NEXT: .LBB0_2: # =>This Inner Loop Header: Depth=1
; CHECK-X86-32-NEXT: xorl $0, (%esp)
; CHECK-X86-32-NEXT: subl $4096, %esp # imm = 0x1000
; CHECK-X86-32-NEXT: cmpl %esp, %eax
-; CHECK-X86-32-NEXT: jl .LBB0_2
+; CHECK-X86-32-NEXT: jb .LBB0_2
; CHECK-X86-32-NEXT: .LBB0_3:
; CHECK-X86-32-NEXT: movl %eax, %esp
; CHECK-X86-32-NEXT: movl $1, 4792(%eax)
diff --git a/llvm/test/CodeGen/X86/stack-clash-small-alloc-medium-align.ll b/llvm/test/CodeGen/X86/stack-clash-small-alloc-medium-align.ll
index ccf7e1d56da90..01a1cb136a49a 100644
--- a/llvm/test/CodeGen/X86/stack-clash-small-alloc-medium-align.ll
+++ b/llvm/test/CodeGen/X86/stack-clash-small-alloc-medium-align.ll
@@ -103,12 +103,12 @@ define i32 @foo4(i64 %i) local_unnamed_addr #0 {
; CHECK-NEXT: andq $-16, %rcx
; CHECK-NEXT: subq %rcx, %rax
; CHECK-NEXT: cmpq %rsp, %rax
-; CHECK-NEXT: jge .LBB3_3
+; CHECK-NEXT: jae .LBB3_3
; CHECK-NEXT: .LBB3_2: # =>This Inner Loop Header: Depth=1
; CHECK-NEXT: xorq $0, (%rsp)
; CHECK-NEXT: subq $4096, %rsp # imm = 0x1000
; CHECK-NEXT: cmpq %rsp, %rax
-; CHECK-NEXT: jl .LBB3_2
+; CHECK-NEXT: jb .LBB3_2
; CHECK-NEXT: .LBB3_3:
; CHECK-NEXT: andq $-64, %rax
; CHECK-NEXT: movq %rax, %rsp
|
Sharjeel-Khan
approved these changes
Apr 15, 2026
Sharjeel-Khan
left a comment
Contributor
There was a problem hiding this comment.
LGTM but I added RKSimon since he works on X86 lowering.
Contributor
|
Did you check if we have the same issue on other non-intel architectures? |
Member
Author
It looks like only I took a quick peek at:
which reference It looks like RISCV 32b has the same issue; it's using |
nickdesaulniers
added a commit
that referenced
this pull request
Apr 16, 2026
The stack clash probing loop generated in `emitDynamicProbedAlloc` used a signed comparison (`RISCV::COND_BLT`) to determine when the allocation target had been reached. In 32-bit mode, memory addresses above `0x80000000` have the sign bit set. If the stack pointer lands in this region, treating the addresses as signed integers causes the comparison logic to fail. This patch changes the condition code to `RISCV::COND_BLTU` (Branch if Less Than Unsigned), which generates an unsigned comparison. This ensures that addresses are treated correctly as unsigned quantities on all targets. On 64-bit systems, this change has no practical effect on valid user-space addresses because they do not use the sign bit (being restricted to the lower half of the address space). However, using unsigned comparison is the correct behavior for pointer arithmetic and bounds checks. Link: #192355
alexfh
pushed a commit
to alexfh/llvm-project
that referenced
this pull request
Apr 18, 2026
The stack clash probing loop generated in `EmitLoweredProbedAlloca` used a signed comparison (`X86::COND_GE`) to determine when the allocation target had been reached. In 32-bit mode, memory addresses above `0x80000000` have the sign bit set. If the stack pointer lands in this region, treating the addresses as signed integers causes the comparison logic to fail. This leads to incorrect loop execution, resulting in an infinite loop and a crash (segmentation fault) when setting up custom stacks for pthreads mapped above `0x80000000` in a 32b process. This patch changes the condition code to `X86::COND_AE` (Above or Equal), which generates an unsigned comparison. This ensures that addresses are treated correctly as unsigned quantities on all targets. On 64-bit systems, this change has no practical effect on valid user-space addresses because they do not use the sign bit (being restricted to the lower half of the address space). However, using unsigned comparison is the correct behavior for pointer arithmetic and bounds checks. Reported-by: Wonsik Kim <wonsik@google.com>
alexfh
pushed a commit
to alexfh/llvm-project
that referenced
this pull request
Apr 18, 2026
…2485) The stack clash probing loop generated in `emitDynamicProbedAlloc` used a signed comparison (`RISCV::COND_BLT`) to determine when the allocation target had been reached. In 32-bit mode, memory addresses above `0x80000000` have the sign bit set. If the stack pointer lands in this region, treating the addresses as signed integers causes the comparison logic to fail. This patch changes the condition code to `RISCV::COND_BLTU` (Branch if Less Than Unsigned), which generates an unsigned comparison. This ensures that addresses are treated correctly as unsigned quantities on all targets. On 64-bit systems, this change has no practical effect on valid user-space addresses because they do not use the sign bit (being restricted to the lower half of the address space). However, using unsigned comparison is the correct behavior for pointer arithmetic and bounds checks. Link: llvm#192355
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The stack clash probing loop generated in
EmitLoweredProbedAllocauseda signed comparison (
X86::COND_GE) to determine when the allocationtarget had been reached.
In 32-bit mode, memory addresses above
0x80000000have the sign bitset. If the stack pointer lands in this region, treating the addresses
as signed integers causes the comparison logic to fail. This leads to
incorrect loop execution, resulting in an infinite loop and a crash
(segmentation fault) when setting up custom stacks for pthreads mapped
above
0x80000000in a 32b process.This patch changes the condition code to
X86::COND_AE(Above orEqual), which generates an unsigned comparison. This ensures that
addresses are treated correctly as unsigned quantities on all targets.
On 64-bit systems, this change has no practical effect on valid
user-space addresses because they do not use the sign bit (being
restricted to the lower half of the address space). However, using
unsigned comparison is the correct behavior for pointer arithmetic and
bounds checks.
Reported-by: Wonsik Kim wonsik@google.com