Skip to content

Update Reqwest to 0.13.1#17543

Closed
salmonsd wants to merge 67 commits intoastral-sh:mainfrom
salmonsd:update-reqwest-tls
Closed

Update Reqwest to 0.13.1#17543
salmonsd wants to merge 67 commits intoastral-sh:mainfrom
salmonsd:update-reqwest-tls

Conversation

@salmonsd
Copy link
Copy Markdown
Contributor

@salmonsd salmonsd commented Jan 16, 2026

Summary

This PR improves the TLS experience by upgrading reqwest to 0.13.1 via #17427

It adds support for three TLS backends via a new hidden --tls-backend flag:

  • rustls-webpki — bundled Mozilla roots from webpki-root-certs (default)
  • rustls — platform/system verifier via rustls-platform-verifier
  • native-tls — native system TLS stack

Custom certificates from SSL_CERT_FILE/SSL_CERT_DIR are merged unconditionally into the root store across all backends using reqwest::tls_merge_certs(), ensuring consistent support in corporate or CI setups without backend-specific gating.

The --native-tls flag and UV_NATIVE_TLS env var are retained for compatibility, mapping to the native-tls backend.

Motivation

reqwest 0.13.1 defaults to rustls as its TLS backend w/ platform verification and removes built-in webpki-roots, and moves its default crypto provider to aws-lc instead of ring (increasing the number of cert signature algos supported) to improve TLS experience.

Changes

  • Dependency updates

  • TLS backend selection

    • Hidden --tls-backend flag: rustls-webpki | rustls | native-tls
    • --native-tls preserved (with explicit conflict handling)
    • UV_NATIVE_TLS env var maps to native-tls backend
    • Default: rustls-webpki
  • Certificate handling

    • Load base roots via webpki-root-certs
    • Use reqwest::tls_certs_only to initialize the root store with bundled certs
    • Merge custom certs from SSL_CERT_FILE/SSL_CERT_DIR using tls_merge_certs
    • Merging is applied unconditionally (no backend gating)
    • Reuses reqwest's certificate merging machinery → avoids custom root store or TLS config management
  • Refactoring & cleanup

    • Centralized logic in uv-client/base_client.rs and uv-client/ssl_certs.rs
    • Removed dead test code
    • Added accept-encoding: identity in registry_client.rs where required
  • Documentation

    • Updated certificates.md:
      • Describes new backends + default
      • Recommends usage patterns (e.g. rustls-webpki for consistency, native-tls for proxies)
      • Explains SSL_CERT_* behavior and migration notes
  • Testing

    • 7 new/expanded tests in uv-client/tests/ssl_certs.rs (loading, precedence, all backends)
    • Updated nextest.toml with SSL test profile override
    • Refreshed subcommand snapshots

Trade-offs & Future Work

  • Using webpki-root-certs + tls_certs_only + tls_merge_certs keeps maintenance low and avoids re-implementing root store logic
  • Manual merging adds control and reduces risk of drift from reqwest internals
  • --native-tls retained for smooth transition; long-term plan is deprecation
  • Planned follow-ups:
    • Promote --tls-backend to visible/stable
    • Introduce --system-certs / --no-system-certs aliases (preview)
    • Switch default to rustls (platform verifier) in a future breaking release
    • Deprecate --native-tls and UV_NATIVE_TLS

@zanieb zanieb self-assigned this Jan 16, 2026
@salmonsd
Copy link
Copy Markdown
Contributor Author

Will work on failing tests (apologies)

@musicinmybrain
Copy link
Copy Markdown
Contributor

We can now also update reqsign to 0.19.0, apache/opendal-reqsign@v0.18.1...v0.19.0, which will remove another user of reqwest 0.12 from the dependency tree.

@salmonsd
Copy link
Copy Markdown
Contributor Author

We can now also update reqsign to 0.19.0, apache/opendal-reqsign@v0.18.1...v0.19.0, which will remove another user of reqwest 0.12 from the dependency tree.

thanks @musicinmybrain, was waiting for the official release!

Only thing to watch for is axoupdater and the update to axoasset (issue here) to also use reqwest 0.13 to be completely resolved.

@salmonsd
Copy link
Copy Markdown
Contributor Author

Hey @zanieb, wanted to check-in and see if y'all have a plan for this or if there's anything I can do to help to get this implemented?

@zanieb zanieb mentioned this pull request Feb 4, 2026
@konstin
Copy link
Copy Markdown
Member

konstin commented Feb 9, 2026

All dependency updates are merged and published.

@salmonsd
Copy link
Copy Markdown
Contributor Author

rebased onto uv 0.10.1 and this commit: 976a368

will work to implement these suggestions: #17543 (comment)

Comment on lines +1138 to +1142
// Specify identity encoding to prevent double compression from async_http_range_reader and reqwest
headers.insert(
reqwest::header::ACCEPT_ENCODING,
reqwest::header::HeaderValue::from_static("identity"),
);
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use tracing::debug;

use uv_client::{BaseClientBuilder, WrappedReqwestError};
use uv_client::BaseClientBuilder;
Copy link
Copy Markdown
Contributor Author

@salmonsd salmonsd Feb 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This removal is related to axoupdater (and depedent axoasset) on an older version of reqwest.

Related:

@salmonsd salmonsd requested review from konstin and zanieb February 18, 2026 04:00
@konstin
Copy link
Copy Markdown
Member

konstin commented Feb 19, 2026

CI is currently failing because we introduced an openssl dependency.

@konstin konstin added the enhancement New feature or improvement to existing functionality label Feb 19, 2026
@zanieb zanieb force-pushed the update-reqwest-tls branch from 9ab78c4 to 6aecc7d Compare March 17, 2026 17:22
zanieb added a commit that referenced this pull request Mar 17, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq bot commented Mar 17, 2026

Merging this PR will not alter performance

❌ 1 regressed benchmark
✅ 4 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime resolve_warm_jupyter_universal 142.3 ms 146.7 ms -3.05%

Comparing salmonsd:update-reqwest-tls (fa1fd36) with main (7b3170d)

Open in CodSpeed

@zanieb zanieb mentioned this pull request Mar 18, 2026
zanieb added a commit that referenced this pull request Mar 23, 2026
The following user-facing changes are included here:

- `aws-lc` is used instead of `ring` for a cryptography backend
- Expands our certificate signature algorithm support to include
ECDSA_P256_SHA512, ECDSA_P384_SHA512, ECDSA_P521_SHA256,
ECDSA_P521_SHA384, and ECDSA_P521_SHA512
- `--native-tls` is deprecated in favor of a new `--system-certs` flag,
avoiding confusion with the TLS implementation used (we use `rustls` not
`native-tls`, see prior confusion at
#11595)
- NASM is a new build requirement on Windows, it is required by `aws-lc`
on x86-64 and i386
- `rustls-platform-verifier` is used instead of `rustls-native-certs`
for system certificate verification
- On macOS, certificate validation is now delegated to
`Security.framework` (`SecTrust`). Performance when using
`--system-certs` is improved by avoiding exporting and parsing all the
certificates from the keychain at startup.
- On Windows, certificate validation is now delegated to
`CertGetCertificateChain` and `CertVerifyCertificateChainPolicy`
    - On Linux, certificate validation should be approximately unchanged
- Some previously failing chains may succeed, and some previously
accepted chains may fail; generally, this should result in behavior
closer matching browsers and other native applications
- macOS and Windows may now perform live OCSP fetches for early
revocation, which could add latency to some requests
- Empty `SSL_CERT_FILE` values are ignored (for consistency with
`SSL_CERT_DIR`)

The following internal changes are included here:

- Certificate loading has been refactored to use a newtype with helper
methods
- The certificate tests have been rewritten
- We use `webpki-root-certs` instead of `webpki-roots`, see
#17543 (comment)
- We request `identity` encoding for range requests, see
astral-sh/async_http_range_reader#3 (comment)
- Various dependencies (including forks) updates to versions which use
reqwest 0.13+

This is a replacement of #17543 with an updated description. See that
pull request for prior discussion. I've made the following changes from
the initial approach there:

- Previously, the `native-tls` TLS implementation was added which
included an OpenSSL build. We don't currently use the `native-tls`
implementation, but the `--native-tls` flag there was erroneously
updated to enable it.
- Previously, there was a `--tls-backend` flag to toggle between
`native-tls` and `rustls`. Since we currently always use `rustls`, this
is deferred to future work (if we need it at all).
- Previously, there were unintentional breaking changes to
`SSL_CERT_FILE` and `SSL_CERT_DIR` handling, including merging with the
base certificates instead of replacing them, dropping support for
OpenSSL hash-named certificate files, skipping deduplication of
certificates. Here, we retain use of `rustls-native-certs` for loading
certificates from the system as it handles these edge cases.


Closes #17427

---------

Co-authored-by: salmonsd <22984014+salmonsd@users.noreply.github.com>
@salmonsd
Copy link
Copy Markdown
Contributor Author

Closing in favor of #18550

@salmonsd salmonsd closed this Mar 23, 2026
@zanieb
Copy link
Copy Markdown
Member

zanieb commented Mar 23, 2026

Thank you for your work on this!

konstin pushed a commit that referenced this pull request Mar 24, 2026
The following user-facing changes are included here:

- `aws-lc` is used instead of `ring` for a cryptography backend
- Expands our certificate signature algorithm support to include
ECDSA_P256_SHA512, ECDSA_P384_SHA512, ECDSA_P521_SHA256,
ECDSA_P521_SHA384, and ECDSA_P521_SHA512
- `--native-tls` is deprecated in favor of a new `--system-certs` flag,
avoiding confusion with the TLS implementation used (we use `rustls` not
`native-tls`, see prior confusion at
#11595)
- NASM is a new build requirement on Windows, it is required by `aws-lc`
on x86-64 and i386
- `rustls-platform-verifier` is used instead of `rustls-native-certs`
for system certificate verification
- On macOS, certificate validation is now delegated to
`Security.framework` (`SecTrust`). Performance when using
`--system-certs` is improved by avoiding exporting and parsing all the
certificates from the keychain at startup.
- On Windows, certificate validation is now delegated to
`CertGetCertificateChain` and `CertVerifyCertificateChainPolicy`
    - On Linux, certificate validation should be approximately unchanged
- Some previously failing chains may succeed, and some previously
accepted chains may fail; generally, this should result in behavior
closer matching browsers and other native applications
- macOS and Windows may now perform live OCSP fetches for early
revocation, which could add latency to some requests
- Empty `SSL_CERT_FILE` values are ignored (for consistency with
`SSL_CERT_DIR`)

The following internal changes are included here:

- Certificate loading has been refactored to use a newtype with helper
methods
- The certificate tests have been rewritten
- We use `webpki-root-certs` instead of `webpki-roots`, see
#17543 (comment)
- We request `identity` encoding for range requests, see
astral-sh/async_http_range_reader#3 (comment)
- Various dependencies (including forks) updates to versions which use
reqwest 0.13+

This is a replacement of #17543 with an updated description. See that
pull request for prior discussion. I've made the following changes from
the initial approach there:

- Previously, the `native-tls` TLS implementation was added which
included an OpenSSL build. We don't currently use the `native-tls`
implementation, but the `--native-tls` flag there was erroneously
updated to enable it.
- Previously, there was a `--tls-backend` flag to toggle between
`native-tls` and `rustls`. Since we currently always use `rustls`, this
is deferred to future work (if we need it at all).
- Previously, there were unintentional breaking changes to
`SSL_CERT_FILE` and `SSL_CERT_DIR` handling, including merging with the
base certificates instead of replacing them, dropping support for
OpenSSL hash-named certificate files, skipping deduplication of
certificates. Here, we retain use of `rustls-native-certs` for loading
certificates from the system as it handles these edge cases.


Closes #17427

---------

Co-authored-by: salmonsd <22984014+salmonsd@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build:release Enable building release binaries for a pull request enhancement New feature or improvement to existing functionality test:extended Enable extended tests for a pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants