Vectorize custom polynomial CRC-32 and CRC-64 by bartonjs · Pull Request #124832 · dotnet/runtime

bartonjs · 2026-02-24T23:08:25Z

With this change, the CRC-32 and CRC-64 computation flows call UpdateVectorized to process as much data as "can" be done (none on .NET Framework, none on Big Endian systems, and up to the last multiple of 128 bits otherwise), then processes the "remainder" with UpdateScalar. UpdateScalar may be table based (most of the time) or a CPU intrinsic (CRC-32/C on x64 or ARM, CRC-32/IEEE on ARM).

This change also brings in more custom polynomial well-known value tests, to corroborate that the polynomial-attached "constants" are being computed correctly.

Fixes #124576.

dotnet-policy-service · 2026-02-24T23:09:07Z

Tagging subscribers to this area: @dotnet/area-system-io-hashing, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Copilot

Pull request overview

This pull request implements vectorized CRC computation for CRC-32 (both forward and reflected variants) and CRC-64 (forward only). The changes refactor the existing vectorization code from the Crc32 and Crc64 classes into the respective ParameterSet classes, enabling vectorization support for custom CRC polynomials.

Changes:

Introduces abstract base classes (ForwardCrc32, ReflectedCrc32, ForwardCrc64) that handle vectorization using a partial method pattern
Adds CrcPolynomialHelper class with a UInt640 struct for computing folding and Barrett constants for arbitrary polynomials
Migrates vectorization logic from Crc32/Crc64 static classes to parameter set implementations

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
CrcPolynomialHelper.cs	New helper class for computing CRC folding/Barrett constants using 640-bit arithmetic
Crc32ParameterSet.cs	Adds `ReflectedCrc32` and `ForwardCrc32` abstract base classes with partial method hooks for vectorization
Crc32ParameterSet.Vectorized.cs	Implements vectorized CRC-32 computation for both reflected and forward variants
Crc32ParameterSet.WellKnown.cs	Updates IEEE 802.3 and CRC-32C to use new base classes with ARM intrinsics support
Crc32ParameterSet.Table.cs	Updates table-based implementations to inherit from new base classes
Crc32.cs	Removes `partial` modifier, delegates to parameter set's Update method
Crc32.Vectorized.cs	Deleted - logic moved to ParameterSet
Crc32.Table.cs	Deleted - table moved to ParameterSet.WellKnown
Crc32.Arm.cs	Deleted - ARM intrinsics moved to ParameterSet.WellKnown
Crc64ParameterSet.cs	Adds `ForwardCrc64` abstract base class with vectorization support
Crc64ParameterSet.Vectorized.cs	Implements vectorized forward CRC-64 computation
Crc64ParameterSet.WellKnown.cs	Updates ECMA-182 to use `ForwardCrc64` base class
Crc64ParameterSet.Table.cs	Updates table-based forward implementation to inherit from `ForwardCrc64`
Crc64.cs	Removes `partial` modifier, delegates to parameter set's Update method
Crc64.Vectorized.cs	Deleted - logic moved to ParameterSet
Crc64.Table.cs	Deleted - table moved to ParameterSet.WellKnown
System.IO.Hashing.csproj	Updates file references to reflect deletions and additions
Crc32Tests_ParameterSet_Custom.cs	Adds test driver for forward CRC-32 with non-standard polynomial

bartonjs · 2026-02-24T23:15:28Z

Perf-wise, it's basically unchanged from the current vectorized implementation for CRC-32/IEEE and CRC-64/ECMA.


BenchmarkDotNet v0.15.8, Windows 11 (10.0.26200.7840/25H2/2025Update/HudsonValley2)
Intel Core i7-8700 CPU 3.20GHz (Max: 3.19GHz) (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET SDK 11.0.100-preview.1.26104.118
  [Host]     : .NET 10.0.3 (10.0.3, 10.0.326.7603), X64 RyuJIT x86-64-v3
  Job-QMWUGV : .NET 11.0.0 (11.0.0-dev, 42.42.42.42424), X64 RyuJIT x86-64-v3
  .NET 10.0  : .NET 10.0.3 (10.0.3, 10.0.326.7603), X64 RyuJIT x86-64-v3

Runtime=.NET 10.0

Method	Job	Toolchain	size	Mean	Error	StdDev	Allocated
Crc32_Prebuilt	Job-QMWUGV	CoreRun	0	2.5973 ns	0.0234 ns	0.0218 ns	-
Crc32_Prebuilt	.NET 10.0	Default	0	2.3792 ns	0.0178 ns	0.0139 ns	-
Crc64_Prebuilt	Job-QMWUGV	CoreRun	0	0.7268 ns	0.0303 ns	0.0283 ns	-
Crc64_Prebuilt	.NET 10.0	Default	0	0.7403 ns	0.0070 ns	0.0062 ns	-
Crc32_Prebuilt	Job-QMWUGV	CoreRun	10	9.6688 ns	0.0179 ns	0.0149 ns	-
Crc32_Prebuilt	.NET 10.0	Default	10	8.8438 ns	0.0230 ns	0.0204 ns	-
Crc64_Prebuilt	Job-QMWUGV	CoreRun	10	8.3621 ns	0.0231 ns	0.0205 ns	-
Crc64_Prebuilt	.NET 10.0	Default	10	8.3531 ns	0.0239 ns	0.0223 ns	-
Crc32_Prebuilt	Job-QMWUGV	CoreRun	100	13.3525 ns	0.0356 ns	0.0333 ns	-
Crc32_Prebuilt	.NET 10.0	Default	100	14.3781 ns	0.0298 ns	0.0279 ns	-
Crc64_Prebuilt	Job-QMWUGV	CoreRun	100	14.8436 ns	0.0454 ns	0.0403 ns	-
Crc64_Prebuilt	.NET 10.0	Default	100	14.7429 ns	0.0191 ns	0.0149 ns	-
Crc32_Prebuilt	Job-QMWUGV	CoreRun	1000	56.0132 ns	0.0887 ns	0.0740 ns	-
Crc32_Prebuilt	.NET 10.0	Default	1000	55.7654 ns	0.0795 ns	0.0744 ns	-
Crc64_Prebuilt	Job-QMWUGV	CoreRun	1000	64.1417 ns	0.0877 ns	0.0778 ns	-
Crc64_Prebuilt	.NET 10.0	Default	1000	63.6507 ns	0.1179 ns	0.1045 ns	-
Crc32_Prebuilt	Job-QMWUGV	CoreRun	10000	347.9694 ns	0.5755 ns	0.5383 ns	-
Crc32_Prebuilt	.NET 10.0	Default	10000	348.6138 ns	0.6414 ns	0.5999 ns	-
Crc64_Prebuilt	Job-QMWUGV	CoreRun	10000	444.4825 ns	0.4317 ns	0.4038 ns	-
Crc64_Prebuilt	.NET 10.0	Default	10000	443.5430 ns	0.6813 ns	0.6373 ns	-
Crc32_Prebuilt	Job-QMWUGV	CoreRun	100000	3,444.3479 ns	5.1116 ns	4.2684 ns	-
Crc32_Prebuilt	.NET 10.0	Default	100000	3,444.2575 ns	3.7278 ns	3.1128 ns	-
Crc64_Prebuilt	Job-QMWUGV	CoreRun	100000	4,402.8019 ns	8.5030 ns	7.9537 ns	-
Crc64_Prebuilt	.NET 10.0	Default	100000	4,401.7223 ns	7.0760 ns	6.2727 ns	-
Crc32_Prebuilt	Job-QMWUGV	CoreRun	1000000	36,523.5935 ns	296.2984 ns	277.1577 ns	-
Crc32_Prebuilt	.NET 10.0	Default	1000000	36,608.2642 ns	236.4533 ns	221.1786 ns	-
Crc64_Prebuilt	Job-QMWUGV	CoreRun	1000000	44,802.2878 ns	72.3593 ns	67.6849 ns	-
Crc64_Prebuilt	.NET 10.0	Default	1000000	44,824.7990 ns	63.7654 ns	56.5264 ns	-
Crc32_Prebuilt	Job-QMWUGV	CoreRun	1048576	38,287.0065 ns	210.2360 ns	186.3687 ns	-
Crc32_Prebuilt	.NET 10.0	Default	1048576	38,268.0418 ns	279.9785 ns	261.8921 ns	-
Crc64_Prebuilt	Job-QMWUGV	CoreRun	1048576	47,016.1375 ns	106.4478 ns	99.5714 ns	-
Crc64_Prebuilt	.NET 10.0	Default	1048576	47,013.3135 ns	74.0468 ns	65.6406 ns	-

src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc32ParameterSet.Vectorized.cs

src/libraries/System.IO.Hashing/src/System/IO/Hashing/CrcPolynomialHelper.cs

tannergooding

SIMD code LGTM and is mostly just a refactoring of the prior logic.

There's some potential future cleanup, but nothing really relevant to the PR here; mostly future work and reducing unnecessary unsafe usage.

Copilot

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.

src/libraries/System.IO.Hashing/src/System/IO/Hashing/CrcPolynomialHelper.cs

src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc64ParameterSet.cs

src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc32ParameterSet.cs

src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc32ParameterSet.WellKnown.cs

bartonjs · 2026-03-07T00:37:48Z

/ba-g The failing tests are #125245, not sure why Build Analysis didn't match it.

bartonjs added 8 commits February 20, 2026 15:38

Forward vectorized CRC-32

1a9cd9e

Vectorized reflected CRC-32

dff310b

Delete old CRC-32 code, defer to parameter-set now.

cd634c4

Vectorize CRC-32/C

9f06747

Vectorized Forward CRC-64

1b3ead1

Delete old CRC-64 code, defer to parameter-set now.

cb68145

Self-Review

e83a15e

Restore the at-least 8-vectors threshold for ARM

e601332

bartonjs added this to the 11.0.0 milestone Feb 24, 2026

bartonjs self-assigned this Feb 24, 2026

bartonjs added the area-System.IO.Hashing label Feb 24, 2026

Copilot AI review requested due to automatic review settings February 24, 2026 23:08

bartonjs requested a review from tannergooding February 24, 2026 23:09

Copilot started reviewing on behalf of bartonjs February 24, 2026 23:09 View session

Copilot AI reviewed Feb 24, 2026

View reviewed changes

build-analysis bot mentioned this pull request Feb 25, 2026

[android][clr] No peer certificates when executing System.Net.Http.Functional.Tests on Android emulator #124526

Open

Add reflected CRC-64

76b5c6e

bartonjs changed the title ~~Vectorize Forward/Reflected CRC-32 and Forward CRC-64~~ Vectorize custom polynomial CRC-32 and CRC-64 Feb 27, 2026

This was referenced Feb 28, 2026

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

[android-arm64] The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#6408

Open

bartonjs mentioned this pull request Mar 4, 2026

Vectorize custom polynomial CRC-32/-64 #124577

Closed

9 tasks

tannergooding reviewed Mar 5, 2026

View reviewed changes

src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc32ParameterSet.Vectorized.cs Outdated Show resolved Hide resolved

tannergooding reviewed Mar 5, 2026

View reviewed changes

src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc32ParameterSet.Vectorized.cs Show resolved Hide resolved

tannergooding reviewed Mar 5, 2026

View reviewed changes

src/libraries/System.IO.Hashing/src/System/IO/Hashing/CrcPolynomialHelper.cs Show resolved Hide resolved

tannergooding reviewed Mar 5, 2026

View reviewed changes

src/libraries/System.IO.Hashing/src/System/IO/Hashing/CrcPolynomialHelper.cs Outdated Show resolved Hide resolved

tannergooding approved these changes Mar 5, 2026

View reviewed changes

Apply feedback

9aa3404

Copilot AI review requested due to automatic review settings March 5, 2026 22:28

Copilot started reviewing on behalf of bartonjs March 5, 2026 22:32 View session

Copilot AI reviewed Mar 5, 2026

View reviewed changes

bartonjs enabled auto-merge (squash) March 5, 2026 22:56

bartonjs added 2 commits March 6, 2026 10:40

Fix comment typo

19c0024

Merge remote-tracking branch 'dotnet/main' into crc_vec3

00d1362

bartonjs merged commit d560dfc into dotnet:main Mar 7, 2026
84 of 90 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Vectorize custom polynomial CRC-32 and CRC-64#124832

Vectorize custom polynomial CRC-32 and CRC-64#124832
bartonjs merged 12 commits intodotnet:mainfrom
bartonjs:crc_vec3

bartonjs commented Feb 24, 2026 •

edited

Loading

Uh oh!

dotnet-policy-service bot commented Feb 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

bartonjs commented Feb 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tannergooding left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bartonjs commented Mar 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

bartonjs commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dotnet-policy-service bot commented Feb 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

bartonjs commented Feb 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tannergooding left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bartonjs commented Mar 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bartonjs commented Feb 24, 2026 •

edited

Loading