Remove deprecated OpenSSL RSA APIs by PranavSenthilnathan · Pull Request #126034 · dotnet/runtime

PranavSenthilnathan · 2026-03-24T15:59:21Z

Remove deprecated OpenSSL RSA APIs. The RSA type and RSA_* methods have been deprecated in favor of EVP_PKEY. Most of our code already has moved, but this cleans up the function loading and updates the remaining places. The build allows deprecated APIs, but for local testing I've been using -DOPENSSL_API_COMPAT=0x30500000L -DOPENSSL_NO_DEPRECATED to error on deprecated APIs.

dotnet-policy-service · 2026-03-24T16:00:29Z

Tagging subscribers to this area: @bartonjs, @vcsjones, @dotnet/area-system-security
See info in area-owners.md if you want to be subscribed.

Copilot

Pull request overview

This PR continues the migration away from deprecated OpenSSL RSA* APIs toward EVP-based APIs (primarily EVP_PKEY), updating certificate generation and key-validation paths, and adjusting shim/configure logic to better tolerate RSA legacy symbol availability.

Changes:

Update self-signed certificate generation to use/generated EVP_PKEY* directly (removing EVP_PKEY_get1_RSA / EVP_PKEY_set1_RSA usage in that flow).
Add an EVP-based RSA “quick check” (EVP_PKEY_get_bn_param) for OpenSSL 3+ to avoid relying on legacy RSA object access.
Add HAVE_OPENSSL_RSA_PRIMITIVE configure/shim support and relax several RSA-related shim bindings to “lightup” (runtime-detected) functions.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/native/libs/System.Security.Cryptography.Native/pal_ssl.c	Switch self-signed cert helper to return the generated `EVP_PKEY` instead of extracting/setting a legacy `RSA`.
src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey_rsa.c	Factor legacy private-key-availability check into a helper and gate it on `EVP_PKEY_get0_RSA` lightup availability.
src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c	Introduce `QuickRsaCheckEvp` using `EVP_PKEY_get_bn_param`; adjust RSA validation flow to prefer EVP-param extraction.
src/native/libs/System.Security.Cryptography.Native/pal_crypto_config.h.in	Add `HAVE_OPENSSL_RSA_PRIMITIVE` feature define.
src/native/libs/System.Security.Cryptography.Native/osslcompat_30.h	Add RSA parameter-name constants needed for EVP param extraction on older headers.
src/native/libs/System.Security.Cryptography.Native/opensslshim.h	Add RSA primitive fallback declarations and convert multiple RSA-related bindings to lightup functions.
src/native/libs/System.Security.Cryptography.Native/configure.cmake	Add compile probe for RSA primitive availability (`RSA_new`/`RSA_free`).

src/native/libs/System.Security.Cryptography.Native/configure.cmake

src/native/libs/System.Security.Cryptography.Native/pal_ssl.c

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c

src/native/libs/System.Security.Cryptography.Native/opensslshim.h

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey_rsa.c

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c

src/native/libs/System.Security.Cryptography.Native/opensslshim.h

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

src/native/libs/System.Security.Cryptography.Native/pal_ssl.c

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c

src/libraries/System.Security.Cryptography/tests/RSACreateTests.cs

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c

bartonjs

The managed to native transition for import/export are in terms of SPKI and PKCS8, so they aren't affected by the loss of RSA_get0_factors and friends.
I think we already don't have good tests for creating an RSAOpenSsl from an RSA*, and I'm having trouble imagining how we'd know it was "OK" to pass a synthetic pointer value in to make sure we got a sensible "uh, there's no EVP_PKEY_set1_RSA...", so it makes sense to not see test changes for that.
All of the new EVP_PKEY vs RSA adapter functions seem to be declared static.
I briefly entertained the idea of suggesting probing off of HEADER_RSA_H instead of probing for RSA_new... but, nah.

So, looks like this "I think you can #if away some more code in direct build mode" is probably the last thing from me.

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey_rsa.c

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c

PranavSenthilnathan · 2026-04-07T21:21:54Z

RSA_F_RSA_NULL_PRIVATE_DECRYPT is 0 is OpenSSL 3.0+ and 132 otherwise. It's used in ERR_put_error for the func argument which has been deprecated (and func doesn't seem very useful anyway). I can use the 132 value but I think it's fine to just remove it altogether here instead of conditionally defining the macros.

bartonjs · 2026-04-07T21:46:47Z

Since the code would only hit when ~~compiled with the 3.0 headers and~~ run on 1.1... it should be passing 132, not 0.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Copilot · 2026-04-07T22:34:54Z

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c

+        if (ERR_peek_error() == 0)
+        {
+            ERR_PUT_error(ERR_LIB_RSA, 0, RSA_R_VALUE_MISSING, __FILE__, __LINE__);
+        }
+


QuickRsaCheckCore treats a getKey(...) failure as a definitive key failure (ret=0) and may set RSA_R_VALUE_MISSING. With the OpenSSL 3.0+ accessor, EVP_PKEY_get_bn_param can fail for provider-managed/opaque keys even when EVP_PKEY_check would succeed. Consider returning -1 (indeterminate) on getKey failure so CheckKey can fall back to EVP_PKEY_check, and reserve definitive failure for the case where required public parameters are confirmed missing.

Suggested change

if (ERR_peek_error() == 0)

{

ERR_PUT_error(ERR_LIB_RSA, 0, RSA_R_VALUE_MISSING, __FILE__, __LINE__);

}

// Some OpenSSL 3.0+ provider-managed or opaque keys cannot expose the RSA

// parameters through getKey() even though EVP_PKEY_check can still validate

// them. Treat this as indeterminate so the caller can fall back.

ret = -1;

Getting n and e should not fail and if they do (e.g. due to OOM) then it's not retryable, so it should be fine to return that the key is invalid.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c

Remove deprecated RSA APIs

c4c843a

PranavSenthilnathan added this to the 11.0.0 milestone Mar 24, 2026

PranavSenthilnathan self-assigned this Mar 24, 2026

Copilot AI review requested due to automatic review settings March 24, 2026 15:59

PranavSenthilnathan added the area-System.Security label Mar 24, 2026

Copilot started reviewing on behalf of PranavSenthilnathan March 24, 2026 16:00 View session

Copilot AI reviewed Mar 24, 2026

View reviewed changes

bartonjs reviewed Mar 25, 2026

View reviewed changes

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey_rsa.c Outdated Show resolved Hide resolved

PranavSenthilnathan added 2 commits April 2, 2026 12:04

Refactor quick check to avoid duplication

148c71f

address comments

a3ecafa

PranavSenthilnathan changed the title ~~[WIP] Remove deprecated OpenSSL RSA APIs~~ Remove deprecated OpenSSL RSA APIs Apr 2, 2026

PranavSenthilnathan marked this pull request as ready for review April 2, 2026 19:57

Copilot AI review requested due to automatic review settings April 2, 2026 19:57

Copilot started reviewing on behalf of PranavSenthilnathan April 2, 2026 19:58 View session

Copilot AI reviewed Apr 2, 2026

View reviewed changes

This was referenced Apr 3, 2026

slow macOS - "##[error]The job running on agent Azure Pipelines 9 ran longer than the maximum time of 60 minutes." dotnet/dnceng#1883

Open

Android arm32 device not found (armeabi-v7a architecture unavailable) #125440

Open

Address comments and add test

9a9cfc5

bartonjs reviewed Apr 3, 2026

View reviewed changes

address comments

39cac2e

Copilot AI review requested due to automatic review settings April 6, 2026 20:32

Copilot started reviewing on behalf of PranavSenthilnathan April 6, 2026 20:33 View session

Copilot AI reviewed Apr 6, 2026

View reviewed changes

PranavSenthilnathan added 2 commits April 6, 2026 13:38

moved up the const macro check out of ifdef

9695540

Don't overwrite error in non-retriable case

17863d7

Copilot AI review requested due to automatic review settings April 6, 2026 21:50

Copilot started reviewing on behalf of PranavSenthilnathan April 6, 2026 21:52 View session

Copilot AI reviewed Apr 6, 2026

View reviewed changes

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c Show resolved Hide resolved

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c Show resolved Hide resolved

Add multiprime check to EVP

fde9110

bartonjs reviewed Apr 7, 2026

View reviewed changes

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c Show resolved Hide resolved

Remove code from nonportable OSSL 1.1/1.0 build

4c1f7af

Copilot AI review requested due to automatic review settings April 7, 2026 17:58

Copilot started reviewing on behalf of PranavSenthilnathan April 7, 2026 18:00 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey_rsa.c Show resolved Hide resolved

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c Show resolved Hide resolved

bartonjs approved these changes Apr 7, 2026

View reviewed changes

Use 0 for func in ERR_PUT_error

93f97d0

Use 132 for RSA_F_RSA_NULL_PRIVATE_DECRYPT in pre-3.0 OSSL

4c0e2bb

Copilot AI review requested due to automatic review settings April 7, 2026 22:29

Copilot started reviewing on behalf of PranavSenthilnathan April 7, 2026 22:30 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

PranavSenthilnathan added 2 commits April 8, 2026 01:40

Only OSSL seems to validate p * q = n

d96c6fa

Merge branch 'main' into ossl-deprecations

001f077

Copilot AI review requested due to automatic review settings April 8, 2026 18:53

Copilot started reviewing on behalf of PranavSenthilnathan April 8, 2026 18:54 View session

Copilot AI reviewed Apr 8, 2026

View reviewed changes

src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey.c Show resolved Hide resolved

Copilot AI mentioned this pull request Apr 11, 2026

Fix WasmTestOnChrome: exclude browser/wasi from gen-debug-dump-docs.py helix post-command #126789

Open

1 task

-        if (ERR_peek_error() == 0)
-        {
-            ERR_PUT_error(ERR_LIB_RSA, 0, RSA_R_VALUE_MISSING, __FILE__, __LINE__);
-        }
+        // Some OpenSSL 3.0+ provider-managed or opaque keys cannot expose the RSA
+        // parameters through getKey() even though EVP_PKEY_check can still validate
+        // them. Treat this as indeterminate so the caller can fall back.
+        ret = -1;

Conversation

PranavSenthilnathan commented Mar 24, 2026

Uh oh!

dotnet-policy-service bot commented Mar 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

bartonjs left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

PranavSenthilnathan commented Apr 7, 2026

Uh oh!

bartonjs commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

PranavSenthilnathan Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

bartonjs commented Apr 7, 2026 •

edited

Loading