Avx2 code causes AccessViolationExceptions with AVX512 available #92357
Description
Description
Avx2 code seems to throw unexpected AccessViolations when AVX512 is available.
Reproduction Steps
- Clone repo https://github.com/TechPizzaDev/SoftwareRasterizer
- Checkout branch net8-upgrade
- Download Castle folder from https://github.com/rawrunprotected/rasterizer/tree/master/SoftwareRasterizer and put next to built SoftwareRasterizer.exe
- Run project in Release mode to crash (works in Debug mode)
Expected behavior
No crash.
Actual behavior
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
at SoftwareRasterizer.Avx2Rasterizer`1[[SoftwareRasterizer.FmaX86, SoftwareRasterizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].rasterizeLoop(UInt32, UInt16*, System.Runtime.Intrinsics.Vector128`1<Single>*, System.Runtime.Intrinsics.Vector128`1<Int32>*, System.Runtime.Intrinsics.Vector128`1<Single>*, System.Runtime.Intrinsics.Vector128`1<Single>*, System.Runtime.Intrinsics.Vector128`1<Single>*, UInt32*, UInt32*, UInt32*, UInt32*, Boolean)
at SoftwareRasterizer.Avx2Rasterizer`1[[SoftwareRasterizer.FmaX86, SoftwareRasterizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].rasterize[[SoftwareRasterizer.Rasterizer+NearClipped, SoftwareRasterizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]](SoftwareRasterizer.Occluder ByRef)
at SoftwareRasterizer.Main.WndProc(TerraFX.Interop.Windows.HWND, UInt32, TerraFX.Interop.Windows.WPARAM, TerraFX.Interop.Windows.LPARAM)
at TerraFX.Interop.Windows.Windows.UpdateWindow(TerraFX.Interop.Windows.HWND)
at TerraFX.Interop.Windows.Windows.UpdateWindow(TerraFX.Interop.Windows.HWND)
at SoftwareRasterizer.Main.wWinMain(TerraFX.Interop.Windows.HINSTANCE)
at SoftwareRasterizer.Program.Main(System.String[])
Regression?
Regression in net8, worked on net7 and net6.
Known Workarounds
Setting DOTNET_EnableAVX512F=0 makes it work as expected in Release mode. The project also works in Debug mode.
Configuration
.NET: 8.0.0-rc.1.23419.4
OS: Microsoft Windows 11 Home (10.0.22621 version 22621)
CPU: 11th Gen Intel(R) Core(TM) i3-1115G4 @ 3.00GHz
Arch: x64
Other information
There seems to be a single codegen difference:
C4E2793DC1 vpmaxsd xmm0, xmm0, xmm1
- C4E2793905EC070000 vpminsd xmm0, xmm0, xmmword ptr [reloc @RWD80] # DOTNET_EnableAVX512F=0
+ 62F27D183905EB070000 vpminsd xmm0, xmm0, dword ptr [reloc @RWD80] {1to4} # DOTNET_EnableAVX512F=1
C5F8298590FCFFFF vmovaps xmmword ptr [rbp-0x370], xmm0DOTNET_EnableAVX512F=0: working disasm.txt
DOTNET_EnableAVX512F=1: broken disasm.txt