[RISCV] Use unsigned comparison for stack clash probing loop#192485
Conversation
The stack clash probing loop generated in `emitDynamicProbedAlloc` used a signed comparison (`RISCV::COND_BLT`) to determine when the allocation target had been reached. In 32-bit mode, memory addresses above `0x80000000` have the sign bit set. If the stack pointer lands in this region, treating the addresses as signed integers causes the comparison logic to fail. This patch changes the condition code to `RISCV::COND_BLTU` (Branch if Less Than Unsigned), which generates an unsigned comparison. This ensures that addresses are treated correctly as unsigned quantities on all targets. On 64-bit systems, this change has no practical effect on valid user-space addresses because they do not use the sign bit (being restricted to the lower half of the address space). However, using unsigned comparison is the correct behavior for pointer arithmetic and bounds checks. Link: llvm#192355
|
@llvm/pr-subscribers-backend-risc-v Author: Nick Desaulniers (nickdesaulniers) ChangesThe stack clash probing loop generated in In 32-bit mode, memory addresses above This patch changes the condition code to On 64-bit systems, this change has no practical effect on valid Link: #192355 Full diff: https://github.com/llvm/llvm-project/pull/192485.diff 3 Files Affected:
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index cdda4604f2188..2a2521c62b8b9 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -26568,8 +26568,8 @@ RISCVTargetLowering::emitDynamicProbedAlloc(MachineInstr &MI,
.addReg(SPReg)
.addImm(0);
- // BLT TargetReg, SP, LoopTest
- BuildMI(*LoopTestMBB, LoopTestMBB->end(), DL, TII->get(RISCV::BLT))
+ // BLTU TargetReg, SP, LoopTest
+ BuildMI(*LoopTestMBB, LoopTestMBB->end(), DL, TII->get(RISCV::BLTU))
.addReg(TargetReg)
.addReg(SPReg)
.addMBB(LoopTestMBB);
diff --git a/llvm/test/CodeGen/RISCV/rvv/stack-probing-dynamic.ll b/llvm/test/CodeGen/RISCV/rvv/stack-probing-dynamic.ll
index f6e257085ee84..1bfc2dece322c 100644
--- a/llvm/test/CodeGen/RISCV/rvv/stack-probing-dynamic.ll
+++ b/llvm/test/CodeGen/RISCV/rvv/stack-probing-dynamic.ll
@@ -29,7 +29,7 @@ define void @dynamic(i64 %size, ptr %out) #0 {
; RV64I-NEXT: .LBB0_1: # =>This Inner Loop Header: Depth=1
; RV64I-NEXT: sub sp, sp, a2
; RV64I-NEXT: sd zero, 0(sp)
-; RV64I-NEXT: blt a0, sp, .LBB0_1
+; RV64I-NEXT: bltu a0, sp, .LBB0_1
; RV64I-NEXT: # %bb.2:
; RV64I-NEXT: mv sp, a0
; RV64I-NEXT: sd a0, 0(a1)
@@ -61,7 +61,7 @@ define void @dynamic(i64 %size, ptr %out) #0 {
; RV32I-NEXT: .LBB0_1: # =>This Inner Loop Header: Depth=1
; RV32I-NEXT: sub sp, sp, a1
; RV32I-NEXT: sw zero, 0(sp)
-; RV32I-NEXT: blt a0, sp, .LBB0_1
+; RV32I-NEXT: bltu a0, sp, .LBB0_1
; RV32I-NEXT: # %bb.2:
; RV32I-NEXT: mv sp, a0
; RV32I-NEXT: sw a0, 0(a2)
@@ -105,7 +105,7 @@ define void @dynamic_fixed(i64 %size, ptr %out1, ptr %out2) #0 {
; RV64I-NEXT: .LBB1_1: # =>This Inner Loop Header: Depth=1
; RV64I-NEXT: sub sp, sp, a1
; RV64I-NEXT: sd zero, 0(sp)
-; RV64I-NEXT: blt a0, sp, .LBB1_1
+; RV64I-NEXT: bltu a0, sp, .LBB1_1
; RV64I-NEXT: # %bb.2:
; RV64I-NEXT: mv sp, a0
; RV64I-NEXT: sd a0, 0(a2)
@@ -139,7 +139,7 @@ define void @dynamic_fixed(i64 %size, ptr %out1, ptr %out2) #0 {
; RV32I-NEXT: .LBB1_1: # =>This Inner Loop Header: Depth=1
; RV32I-NEXT: sub sp, sp, a1
; RV32I-NEXT: sw zero, 0(sp)
-; RV32I-NEXT: blt a0, sp, .LBB1_1
+; RV32I-NEXT: bltu a0, sp, .LBB1_1
; RV32I-NEXT: # %bb.2:
; RV32I-NEXT: mv sp, a0
; RV32I-NEXT: sw a0, 0(a3)
@@ -188,7 +188,7 @@ define void @dynamic_align_64(i64 %size, ptr %out) #0 {
; RV64I-NEXT: .LBB2_1: # =>This Inner Loop Header: Depth=1
; RV64I-NEXT: sub sp, sp, a2
; RV64I-NEXT: sd zero, 0(sp)
-; RV64I-NEXT: blt a0, sp, .LBB2_1
+; RV64I-NEXT: bltu a0, sp, .LBB2_1
; RV64I-NEXT: # %bb.2:
; RV64I-NEXT: mv sp, a0
; RV64I-NEXT: sd a0, 0(a1)
@@ -227,7 +227,7 @@ define void @dynamic_align_64(i64 %size, ptr %out) #0 {
; RV32I-NEXT: .LBB2_1: # =>This Inner Loop Header: Depth=1
; RV32I-NEXT: sub sp, sp, a1
; RV32I-NEXT: sw zero, 0(sp)
-; RV32I-NEXT: blt a0, sp, .LBB2_1
+; RV32I-NEXT: bltu a0, sp, .LBB2_1
; RV32I-NEXT: # %bb.2:
; RV32I-NEXT: mv sp, a0
; RV32I-NEXT: sw a0, 0(a2)
@@ -287,7 +287,7 @@ define void @dynamic_align_8192(i64 %size, ptr %out) #0 {
; RV64I-NEXT: .LBB3_1: # =>This Inner Loop Header: Depth=1
; RV64I-NEXT: sub sp, sp, a2
; RV64I-NEXT: sd zero, 0(sp)
-; RV64I-NEXT: blt a0, sp, .LBB3_1
+; RV64I-NEXT: bltu a0, sp, .LBB3_1
; RV64I-NEXT: # %bb.2:
; RV64I-NEXT: mv sp, a0
; RV64I-NEXT: sd a0, 0(a1)
@@ -338,7 +338,7 @@ define void @dynamic_align_8192(i64 %size, ptr %out) #0 {
; RV32I-NEXT: .LBB3_1: # =>This Inner Loop Header: Depth=1
; RV32I-NEXT: sub sp, sp, a1
; RV32I-NEXT: sw zero, 0(sp)
-; RV32I-NEXT: blt a0, sp, .LBB3_1
+; RV32I-NEXT: bltu a0, sp, .LBB3_1
; RV32I-NEXT: # %bb.2:
; RV32I-NEXT: mv sp, a0
; RV32I-NEXT: sw a0, 0(a2)
@@ -382,7 +382,7 @@ define void @no_reserved_call_frame(i64 %n) #0 {
; RV64I-NEXT: # =>This Inner Loop Header: Depth=1
; RV64I-NEXT: sub sp, sp, a1
; RV64I-NEXT: sd zero, 0(sp)
-; RV64I-NEXT: blt a0, sp, .LBB4_1
+; RV64I-NEXT: bltu a0, sp, .LBB4_1
; RV64I-NEXT: # %bb.2: # %entry
; RV64I-NEXT: mv sp, a0
; RV64I-NEXT: lui a1, 1
@@ -421,7 +421,7 @@ define void @no_reserved_call_frame(i64 %n) #0 {
; RV32I-NEXT: # =>This Inner Loop Header: Depth=1
; RV32I-NEXT: sub sp, sp, a1
; RV32I-NEXT: sw zero, 0(sp)
-; RV32I-NEXT: blt a0, sp, .LBB4_1
+; RV32I-NEXT: bltu a0, sp, .LBB4_1
; RV32I-NEXT: # %bb.2: # %entry
; RV32I-NEXT: mv sp, a0
; RV32I-NEXT: lui a1, 1
@@ -538,7 +538,7 @@ define void @dynamic_vector(i64 %size, ptr %out) #0 {
; RV64I-NEXT: .LBB6_1: # =>This Inner Loop Header: Depth=1
; RV64I-NEXT: sub sp, sp, a2
; RV64I-NEXT: sd zero, 0(sp)
-; RV64I-NEXT: blt a0, sp, .LBB6_1
+; RV64I-NEXT: bltu a0, sp, .LBB6_1
; RV64I-NEXT: # %bb.2:
; RV64I-NEXT: mv sp, a0
; RV64I-NEXT: sd a0, 0(a1)
@@ -571,7 +571,7 @@ define void @dynamic_vector(i64 %size, ptr %out) #0 {
; RV32I-NEXT: .LBB6_1: # =>This Inner Loop Header: Depth=1
; RV32I-NEXT: sub sp, sp, a1
; RV32I-NEXT: sw zero, 0(sp)
-; RV32I-NEXT: blt a0, sp, .LBB6_1
+; RV32I-NEXT: bltu a0, sp, .LBB6_1
; RV32I-NEXT: # %bb.2:
; RV32I-NEXT: mv sp, a0
; RV32I-NEXT: sw a0, 0(a2)
diff --git a/llvm/test/CodeGen/RISCV/stack-clash-prologue.ll b/llvm/test/CodeGen/RISCV/stack-clash-prologue.ll
index b76f2b2464709..da11dae07dcca 100644
--- a/llvm/test/CodeGen/RISCV/stack-clash-prologue.ll
+++ b/llvm/test/CodeGen/RISCV/stack-clash-prologue.ll
@@ -650,7 +650,7 @@ define void @f11(i32 %vla_size, i64 %i) #0 {
; RV64I-NEXT: .LBB11_3: # =>This Inner Loop Header: Depth=1
; RV64I-NEXT: sub sp, sp, a1
; RV64I-NEXT: sd zero, 0(sp)
-; RV64I-NEXT: blt a0, sp, .LBB11_3
+; RV64I-NEXT: bltu a0, sp, .LBB11_3
; RV64I-NEXT: # %bb.4:
; RV64I-NEXT: mv sp, a0
; RV64I-NEXT: lbu zero, 0(a0)
@@ -707,7 +707,7 @@ define void @f11(i32 %vla_size, i64 %i) #0 {
; RV32I-NEXT: .LBB11_3: # =>This Inner Loop Header: Depth=1
; RV32I-NEXT: sub sp, sp, a1
; RV32I-NEXT: sw zero, 0(sp)
-; RV32I-NEXT: blt a0, sp, .LBB11_3
+; RV32I-NEXT: bltu a0, sp, .LBB11_3
; RV32I-NEXT: # %bb.4:
; RV32I-NEXT: mv sp, a0
; RV32I-NEXT: lbu zero, 0(a0)
|
…2485) The stack clash probing loop generated in `emitDynamicProbedAlloc` used a signed comparison (`RISCV::COND_BLT`) to determine when the allocation target had been reached. In 32-bit mode, memory addresses above `0x80000000` have the sign bit set. If the stack pointer lands in this region, treating the addresses as signed integers causes the comparison logic to fail. This patch changes the condition code to `RISCV::COND_BLTU` (Branch if Less Than Unsigned), which generates an unsigned comparison. This ensures that addresses are treated correctly as unsigned quantities on all targets. On 64-bit systems, this change has no practical effect on valid user-space addresses because they do not use the sign bit (being restricted to the lower half of the address space). However, using unsigned comparison is the correct behavior for pointer arithmetic and bounds checks. Link: llvm#192355
…llFramePseudoInstr` (llvm#195456) [ Upstream commit 589faed ] Revert `bltu` in probing loops to `blt` because commit f162be2 isn't applied on release/22.x yet. Link: llvm#192485 ("[RISCV] Use unsigned comparison for stack clash probing loop") --- This PR adds a call to `inlineStackProbe` immediately after `allocateStack` in `eliminateCallFramePseudoInstr`. This allows code generation for stack probe pseudoinstructions in non-entry BBs. Fixes llvm#195454.
…eliminateCallFramePseudoInstr` (#195456) [ Upstream commit 589faed ] Revert `bltu` in probing loops to `blt` because commit f162be2 isn't applied on release/22.x yet. Link: llvm/llvm-project#192485 ("[RISCV] Use unsigned comparison for stack clash probing loop") --- This PR adds a call to `inlineStackProbe` immediately after `allocateStack` in `eliminateCallFramePseudoInstr`. This allows code generation for stack probe pseudoinstructions in non-entry BBs. Fixes #195454.
…llFramePseudoInstr` (llvm#195456) [ Upstream commit 589faed ] Revert `bltu` in probing loops to `blt` because commit f162be2 isn't applied on release/22.x yet. Link: llvm#192485 ("[RISCV] Use unsigned comparison for stack clash probing loop") --- This PR adds a call to `inlineStackProbe` immediately after `allocateStack` in `eliminateCallFramePseudoInstr`. This allows code generation for stack probe pseudoinstructions in non-entry BBs. Fixes llvm#195454.
The stack clash probing loop generated in
emitDynamicProbedAllocuseda signed comparison (
RISCV::COND_BLT) to determine when the allocationtarget had been reached.
In 32-bit mode, memory addresses above
0x80000000have the sign bitset. If the stack pointer lands in this region, treating the addresses
as signed integers causes the comparison logic to fail.
This patch changes the condition code to
RISCV::COND_BLTU(Branch ifLess Than Unsigned), which generates an unsigned comparison. This
ensures that addresses are treated correctly as unsigned quantities on
all targets.
On 64-bit systems, this change has no practical effect on valid
user-space addresses because they do not use the sign bit (being
restricted to the lower half of the address space). However, using
unsigned comparison is the correct behavior for pointer arithmetic and
bounds checks.
Link: #192355