Support transparent Python patch version upgrades #13954

jtfmumm · 2025-06-10T19:47:38Z

NOTE: The PRs that were merged into this feature branch have all been independently reviewed. But it's also useful to see all of the changes in their final form. I've added comments to significant changes throughout the PR to aid discussion.

This PR introduces transparent Python version upgrades to uv, allowing for a smoother experience when upgrading to new patch versions. Previously, upgrading Python patch versions required manual updates to each virtual environment. Now, virtual environments can transparently upgrade to newer patch versions.

Due to significant changes in how uv installs and executes managed Python executables, this functionality is initially available behind a --preview flag. Once an installation has been made upgradeable through --preview, subsequent operations (like uv venv -p 3.10 or patch upgrades) will work without requiring the flag again. This is accomplished by checking for the existence of a minor version symlink directory (or junction on Windows).

Features

New uv python upgrade command to upgrade installed Python versions to the latest available patch release:

# Upgrade specific minor version 
uv python upgrade 3.12 --preview
# Upgrade all installed minor versions
uv python upgrade --preview

Transparent upgrades also occur when installing newer patch versions:

uv python install 3.10.8 --preview
# Automatically upgrades existing 3.10 environments
uv python install 3.10.18

Support for transparently upgradeable Python bin installations via --preview flag

uv python install 3.13 --preview
# Automatically upgrades the `bin` installation if there is a newer patch version available
uv python upgrade 3.13 --preview

Virtual environments can still be tied to a patch version if desired (ignoring patch upgrades):

uv venv -p 3.10.8

Implementation

Transparent upgrades are implemented using:

Minor version symlink directories (Unix) or junctions (Windows)
On Windows, trampolines simulate paths with junctions
Symlink directory naming follows Python build standalone format: e.g., cpython-3.10-macos-aarch64-none
Upgrades are scoped to the minor version key (as represented in the naming format: implementation-minor version+variant-os-arch-libc)
If the context does not provide a patch version request and the interpreter is from a managed CPython installation, the Interpreter used by uv python run will use the full symlink directory executable path when available, enabling transparently upgradeable environments created with the venv module (uv run python -m venv)

New types:

PythonMinorVersionLink: in a sense, the core type for this PR, this is a representation of a minor version symlink directory (or junction on Windows) that points to the highest installed managed CPython patch version for a minor version key.
PythonInstallationMinorVersionKey: provides a view into a PythonInstallationKey that excludes the patch and prerelease. This is used for grouping installations by minor version key (e.g., to find the highest available patch installation for that minor version key) and for minor version directory naming.

Compatibility

Supports virtual environments created with:
- uv venv
- uv run python -m venv (using managed Python that was installed or upgraded with --preview)
- Virtual environments created within these environments
Existing virtual environments from before these changes continue to work but aren't transparently upgradeable without being recreated
Supports both standard Python (python3.10) and freethreaded Python (python3.10t)
Support for transparently upgrades is currently only available for managed CPython installations

Closes #7287
Closes #7325
Closes #7892
Closes #9031
Closes #12977

jtfmumm · 2025-06-10T19:51:20Z

crates/uv-python/src/discovery.rs

+                            PythonSource::Managed,
+                            if version.patch().is_some() || is_alternative_implementation {
+                                installation.executable(false)
+                            } else {


When deriving an executable from an installation, if the version is not a patch and it's a CPython installation, uv looks for a minor version symlink directory. If one is found, it will use the symlink-directory containing path. This enables transparently upgradeable virtual environments created with the venv module.

jtfmumm · 2025-06-10T19:55:48Z

crates/uv-python/src/installation.rs

+            .patch()
+            .is_some_and(|p| p >= highest_patch)
+        {
+            installed.ensure_minor_version_link(preview)?;


uv checks for the highest patch for each minor version key and ensures that the corresponding minor version link exists and points to it (but a symlink directory is only initially created in preview mode).

jtfmumm · 2025-06-10T19:56:43Z

crates/uv-python/src/installation.rs

+/// A view into a [`PythonInstallationKey`] that excludes the patch and prerelease versions.
+#[derive(Clone, Eq, Ord, PartialOrd, RefCast)]
+#[repr(transparent)]
+pub struct PythonInstallationMinorVersionKey(PythonInstallationKey);


Used for grouping by the full key minus the patch and prerelease. Also used for deriving symlink directory names.

jtfmumm · 2025-06-10T19:58:15Z

crates/uv-python/src/interpreter.rs


+/// Calls `fs_err::canonicalize` on Unix. On Windows, avoids attempting to resolve symlinks
+/// but will resolve junctions if they are part of a trampoline target.
+pub fn canonicalize_executable(path: impl AsRef<Path>) -> std::io::Result<PathBuf> {


This has been moved from uv_fs because it now knows about trampolines. If the path is to a trampoline, this will canonicalize the trampoline's internal Python path. If not, it will behave as before.

crates/uv-python/src/managed.rs

jtfmumm · 2025-06-10T20:05:53Z

crates/uv-trampoline/src/bounce.rs

    // Only execute the trampoline again if it's a script, otherwise, just invoke Python.
    match kind {
-        TrampolineKind::Python => {}
+        TrampolineKind::Python => {


Updates to the trampoline logic to set the __PYVENV_LAUNCHER__ env var for a Python launcher (the approach taken by CPython for Python Launchers) and, if it's not a virtual environment and the PYTHONHOME is not set, to set the PYTHONHOME env var to ensure the correct directories are added to sys.path when running with a junction trampoline.

jtfmumm · 2025-06-10T20:07:17Z

crates/uv-trampoline/src/bounce.rs

-    let path = dunce::canonicalize(path.as_path()).unwrap_or_else(|_| {
-        error_and_exit("Failed to canonicalize script path");
-    });
+    let path = if !path.is_absolute() || matches!(kind, TrampolineKind::Script) {


This update is required so we don't resolve junctions for junction trampolines. We need to be able to use the junction to ensure transparent upgrades.

jtfmumm · 2025-06-10T20:10:16Z

crates/uv-virtualenv/src/virtualenv.rs

    // Create a `.gitignore` file to ignore all files in the venv.
    fs::write(location.join(".gitignore"), "*")?;

+    let executable_target = if upgradeable && interpreter.is_standalone() {


If this is an upgradeable request, we'll try to use a Python minor version link when creating the virtual environment (creating the symlink directory/junction if it doesn't exist).

jtfmumm · 2025-06-10T20:12:27Z

crates/uv/src/commands/python/install.rs

-        .into_iter()
-        .map(|a| InstallRequest::new(a, python_downloads_json_url.as_deref()))
-        .collect::<Result<Vec<_>>>()?
+        if upgrade {


If this is an upgrade with empty targets, uv collects minor version requests.

jtfmumm · 2025-06-10T20:14:48Z

crates/uv/src/commands/python/install.rs

+            existing_installations,
+        );
+
+    for installation in minor_versions.values() {


Ensure the minor version links exist for the highest patch installations per minor version key.

jtfmumm · 2025-06-10T20:15:58Z

crates/uv/src/commands/python/install.rs

    for target in targets {
        let target = bin.join(target);
-        match installation.create_bin_link(&target) {
+        let executable = if upgradeable {


When creating bin links, we now try to use a symlink directory as the bin link target if we can and this is an upgradeable request.

jtfmumm · 2025-06-10T20:17:10Z

crates/uv/src/commands/python/uninstall.rs

+    // For each uninstalled installation, check if there are no remaining installations
+    // for its minor version. If there are none remaining, remove the symlink directory
+    // (or junction on Windows) if it exists.
+    for installation in &matching_installations {


Here's where we uninstall symlink directories if there are no installations remaining for the corresponding minor version key.

jtfmumm · 2025-06-10T20:18:22Z

crates/uv/src/lib.rs

+            let reinstall = false;
+            let upgrade = true;
+
+            commands::python_install(


I had initially tried creating a new python_upgrade command, but there is so much shared machinery that it seemed like an unnecessary amount of maintenance overhead to split the code paths. Sharing them also makes adding python install --upgrade trivial if we decide to go that route.

crates/uv-python/src/discovery.rs

zanieb · 2025-06-11T22:50:03Z

crates/uv-python/src/interpreter.rs

+    let path = path.as_ref();
+    debug_assert!(
+        path.is_absolute(),
+        "path must be absolute: {}",
+        path.display()
+    );


Not a part of this pull request, but we should follow up on it.

I'm still confused by this, why must the path be absolute? That isn't documented for this function. I think I asked this in a previous pull request.

Yeah we can follow up on it. This was a pre-existing invariant and my changes didn't eliminate the cases it would have applied to.

iitc if we get a relative path here we don't know what it would be relative too (CWD, PYTHONHOME or something else), so we chose to make this the responsibility of the caller. We're now passing sys_executable and sys_base_executable here, which are not enforced to be absolute, so we should change this from a panic to an error.

crates/uv-virtualenv/src/virtualenv.rs

crates/uv-python/src/managed.rs

…Link` (#13953) The `DirectorySymlink` struct assumes that it will be a Python minor version link, so this renaming makes the intention clearer.

…anch (#13965) The prerelease bump on `main` was lost when rebasing `feature/transparent-python-upgrades` against it.

This PR: * renames `PythonMinorVersionLink::symlink_exists()` to `..::exists` * avoids rediscovering installations by chaining together new installations with pre-existing installations when searching for the highest patch per minor version key * renames a couple of related errors and updates their messages to be more specific * renames `create_bin_link` to `create_link_to_executable` * removes a constructor that is no longer used (`PythonMinorVersinLink::from_interpreter()`)

This PR orders minor version requests and highest installation by minor version keys by insertion order (using `indexmap::IndexSet` and `IndexMap`). It also includes a couple of minor style updates.

…n` installation (#14084) `uv python upgrade` had originally had a `--force` option, but we thought it probably didn't make sense. However, switching `uv python upgrade` behind the preview flag revealed a bad interaction: upgrading a minor version when a non-managed interpreter of that version exists in the `bin` directory displays an error recommending the use of `--force`. This PR only warns if this case arises with `uv python upgrade`, noting that install `--force` can be used. It also includes a test for the case described above.

…ries (#14131) For the preview version of transparent upgrades, this PR helps to more clearly distinguish two features: (1) upgrading by installing the latest patch version for a minor version and (2) supporting transparent upgrades for things like virtual environments. Transparently upgradeable installations now require `uv python install` with the `--preview` flag. `uv python upgrade` will no longer create symlink directories on its own, only point one to the latest patch if it already exists. This PR also changes `uv venv --preview` to no longer create a missing symlink directory (a behavior that was meant to support a smooth transition when upgrading uv, but which doesn't make sense in light of the core change of this PR).

Some edits for #13954

neutrinoceros · 2025-06-20T16:48:14Z

Hi, I think I authored the oldest request for this feature (#7287) and I must say, I'm amazed by the amount of work that went into it. I had no idea it would be that involved, and I'm eternally grateful for your effort. Thank you so much !

callegar · 2025-06-21T09:22:27Z

Will this also be available for the tool subcommand that internally relies on virtualenvs. Specifically, will it be possible to say uv tool update -p <something>, or uv tool update -p <something> --all?

Even more interesting: in case you upgrade a managed python minor version (e.g uv python install 3.15.3 and then uv python uninstall 3.15.2) will it then be possible to have uv tool update --all update all the virtualenvs using the not-anymore-existing minor python version to the newer one?

This reverts commit e9d5780.

…lations.find_all()` (#14180) #13954 introduced an unnecessary slow-down to Python uninstall by calling `installations.find_all()` to discover remaining installations after an uninstall. Instead, we can filter all initial installations against those in `uninstalled`. As part of this change, I've updated `uninstalled` from a `Vec` to an `IndexSet` in order to do efficient lookups in the filter. This required a change I call out below to how we were retrieving them for messaging.

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.7.13` -> `0.7.14` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.7.14`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0714) [Compare Source](astral-sh/uv@0.7.13...0.7.14) ##### Enhancements - Add XPU to `--torch-backend` ([#14172](astral-sh/uv#14172)) - Add ROCm backends to `--torch-backend` ([#14120](astral-sh/uv#14120)) - Remove preview label from `--torch-backend` ([#14119](astral-sh/uv#14119)) - Add `[tool.uv.dependency-groups].mygroup.requires-python` ([#13735](astral-sh/uv#13735)) - Add auto-detection for AMD GPUs ([#14176](astral-sh/uv#14176)) - Show retries for HTTP status code errors ([#13897](astral-sh/uv#13897)) - Support transparent Python patch version upgrades ([#13954](astral-sh/uv#13954)) - Warn on empty index directory ([#13940](astral-sh/uv#13940)) - Publish to DockerHub ([#14088](astral-sh/uv#14088)) ##### Performance - Make cold resolves about 10% faster ([#14035](astral-sh/uv#14035)) ##### Bug fixes - Don't use walrus operator in interpreter query script ([#14108](astral-sh/uv#14108)) - Fix handling of changes to `requires-python` ([#14076](astral-sh/uv#14076)) - Fix implied `platform_machine` marker for `win_amd64` platform tag ([#14041](astral-sh/uv#14041)) - Only update existing symlink directories on preview uninstall ([#14179](astral-sh/uv#14179)) - Serialize Python requests for tools as canonicalized strings ([#14109](astral-sh/uv#14109)) - Support netrc and same-origin credential propagation on index redirects ([#14126](astral-sh/uv#14126)) - Support reading `dependency-groups` from pyproject.tomls with no `[project]` ([#13742](astral-sh/uv#13742)) - Handle an existing shebang in `uv init --script` ([#14141](astral-sh/uv#14141)) - Prevent concurrent updates of the environment in `uv run` ([#14153](astral-sh/uv#14153)) - Filter managed Python distributions by platform before querying when included in request ([#13936](astral-sh/uv#13936)) ##### Documentation - Replace cuda124 with cuda128 ([#14168](astral-sh/uv#14168)) - Document the way member sources shadow workspace sources ([#14136](astral-sh/uv#14136)) - Sync documented PyTorch integration index for CUDA and ROCm versions from PyTorch website ([#14100](astral-sh/uv#14100)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

This reverts commit e9d5780.

#14252) There was a regression introduced in #13954 on Windows where creating a venv behaved as if there was a minor version link even if none existed. This PR adds a check to fix this. Closes #14249.

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [ghcr.io/astral-sh/uv](https://github.com/astral-sh/uv) | final | patch | `0.7.13` -> `0.7.15` | --- ### Release Notes <details> <summary>astral-sh/uv (ghcr.io/astral-sh/uv)</summary> ### [`v0.7.15`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0715) [Compare Source](astral-sh/uv@0.7.14...0.7.15) ##### Enhancements - Consistently use `Ordering::Relaxed` for standalone atomic use cases ([#14190](astral-sh/uv#14190)) - Warn on ambiguous relative paths for `--index` ([#14152](astral-sh/uv#14152)) - Skip GitHub fast path when rate-limited ([#13033](astral-sh/uv#13033)) - Preserve newlines in `schema.json` descriptions ([#13693](astral-sh/uv#13693)) ##### Bug fixes - Add check for using minor version link when creating a venv on Windows ([#14252](astral-sh/uv#14252)) - Strip query parameters when parsing source URL ([#14224](astral-sh/uv#14224)) ##### Documentation - Add a link to PyPI FAQ to clarify what per-project token is ([#14242](astral-sh/uv#14242)) ##### Preview features - Allow symlinks in the build backend ([#14212](astral-sh/uv#14212)) ### [`v0.7.14`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0714) [Compare Source](astral-sh/uv@0.7.13...0.7.14) ##### Enhancements - Add XPU to `--torch-backend` ([#14172](astral-sh/uv#14172)) - Add ROCm backends to `--torch-backend` ([#14120](astral-sh/uv#14120)) - Remove preview label from `--torch-backend` ([#14119](astral-sh/uv#14119)) - Add `[tool.uv.dependency-groups].mygroup.requires-python` ([#13735](astral-sh/uv#13735)) - Add auto-detection for AMD GPUs ([#14176](astral-sh/uv#14176)) - Show retries for HTTP status code errors ([#13897](astral-sh/uv#13897)) - Support transparent Python patch version upgrades ([#13954](astral-sh/uv#13954)) - Warn on empty index directory ([#13940](astral-sh/uv#13940)) - Publish to DockerHub ([#14088](astral-sh/uv#14088)) ##### Performance - Make cold resolves about 10% faster ([#14035](astral-sh/uv#14035)) ##### Bug fixes - Don't use walrus operator in interpreter query script ([#14108](astral-sh/uv#14108)) - Fix handling of changes to `requires-python` ([#14076](astral-sh/uv#14076)) - Fix implied `platform_machine` marker for `...

jtfmumm temporarily deployed to release June 10, 2025 19:47 — with GitHub Actions Inactive

jtfmumm commented Jun 10, 2025

View reviewed changes

crates/uv-python/src/managed.rs Show resolved Hide resolved

jtfmumm commented Jun 10, 2025

View reviewed changes

jtfmumm marked this pull request as ready for review June 10, 2025 20:27

jtfmumm requested review from konstin and zanieb June 10, 2025 21:00

jtfmumm force-pushed the feature/transparent-python-upgrades branch from 942f47c to 56b5fda Compare June 11, 2025 13:22

jtfmumm temporarily deployed to release June 11, 2025 13:22 — with GitHub Actions Inactive

jtfmumm temporarily deployed to release June 11, 2025 13:53 — with GitHub Actions Inactive

zanieb reviewed Jun 11, 2025

View reviewed changes

crates/uv-python/src/discovery.rs Outdated Show resolved Hide resolved

zanieb reviewed Jun 11, 2025

View reviewed changes

crates/uv-python/src/discovery.rs Show resolved Hide resolved

zanieb reviewed Jun 11, 2025

View reviewed changes

crates/uv-virtualenv/src/virtualenv.rs Outdated Show resolved Hide resolved

jtfmumm commented Jun 11, 2025

View reviewed changes

crates/uv-python/src/managed.rs Outdated Show resolved Hide resolved

jtfmumm temporarily deployed to release June 11, 2025 23:59 — with GitHub Actions Inactive

jtfmumm and others added 8 commits June 20, 2025 15:32

Rename DirectorySymlink to the more descriptive `PythonMinorVersion…

6098ef7

…Link` (#13953) The `DirectorySymlink` struct assumes that it will be a Python minor version link, so this renaming makes the intention clearer.

Update prerelease in python_install_314 test on upgrades feature br…

ecaf338

…anch (#13965) The prerelease bump on `main` was lost when rebasing `feature/transparent-python-upgrades` against it.

Simplify Python version link construction in discovery (#13980)

776c025

Refactors to transparent python upgrades (#14101)

33b5fd2

This PR orders minor version requests and highest installation by minor version keys by insertion order (using `indexmap::IndexSet` and `IndexMap`). It also includes a couple of minor style updates.

Change upgrade preview error into a warning

9c039d3

jtfmumm force-pushed the feature/transparent-python-upgrades branch from 33adf2b to 4478db3 Compare June 20, 2025 13:32

jtfmumm temporarily deployed to release June 20, 2025 13:32 — with GitHub Actions Inactive

jtfmumm temporarily deployed to uv-test-registries June 20, 2025 13:34 — with GitHub Actions Inactive

Improvements to the Python upgrade documentation (#14132)

7983bbf

Some edits for #13954

jtfmumm temporarily deployed to release June 20, 2025 13:40 — with GitHub Actions Inactive

jtfmumm temporarily deployed to uv-test-registries June 20, 2025 13:42 — with GitHub Actions Inactive

jtfmumm merged commit e9d5780 into main Jun 20, 2025
110 checks passed

jtfmumm deleted the feature/transparent-python-upgrades branch June 20, 2025 14:17

johnthagen mentioned this pull request Jun 20, 2025

Document uv python upgrade johnthagen/python-blueprint#290

Open

Gankra mentioned this pull request Jun 20, 2025

test pip_uninstall::uninstall* times out on Windows (has been running for over 60 seconds) #14158

Closed

jtfmumm mentioned this pull request Jun 21, 2025

Improve Python uninstall perf by removing unnecessary call to installations.find_all() #14180

Merged

jtfmumm added a commit that referenced this pull request Jun 21, 2025

Revert "Support transparent Python patch version upgrades (#13954)"

f09b2fd

This reverts commit e9d5780.

BrewTestBot mentioned this pull request Jun 23, 2025

uv 0.7.14 Homebrew/homebrew-core#227918

Merged

jtfmumm added a commit that referenced this pull request Jun 25, 2025

Revert "Support transparent Python patch version upgrades (#13954)"

03cac0d

This reverts commit e9d5780.

jtfmumm mentioned this pull request Jun 25, 2025

Add check for using minor version link when creating a venv on Windows #14252

Merged

zanieb mentioned this pull request Jul 8, 2025

uv tool install --reinstall does not preserve packages injected with --with #13695

Closed

Support transparent Python patch version upgrades #13954

Support transparent Python patch version upgrades #13954

Uh oh!

Conversation

jtfmumm commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Features

Implementation

Compatibility

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jtfmumm Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

neutrinoceros commented Jun 20, 2025

Uh oh!

callegar commented Jun 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jtfmumm commented Jun 10, 2025 •

edited

Loading

jtfmumm Jun 10, 2025 •

edited

Loading