Skip to content

Replace virtualenv with python -m venv in virtualenv env_manager path#20640

Merged
harupy merged 7 commits intomasterfrom
copilot/replace-virtualenv-with-venv
Feb 8, 2026
Merged

Replace virtualenv with python -m venv in virtualenv env_manager path#20640
harupy merged 7 commits intomasterfrom
copilot/replace-virtualenv-with-venv

Conversation

Copy link
Contributor

Copilot AI commented Feb 7, 2026

Related Issues/PRs

N/A

What changes are proposed in this pull request?

Replaces the third-party virtualenv package with Python's built-in venv module for the env_manager="virtualenv" code path. The user-facing option name remains "virtualenv" for backward compatibility.

Core changes:

  • mlflow/utils/virtualenv.py: Changed environment creation from sys.executable -m virtualenv --python <target> to <target> -m venv
  • Removed _is_virtualenv_available() and _validate_virtualenv_is_available() validation functions
  • Removed virtualenv from pyproject.toml (sagemaker extras) and dev/pyproject.py

Infrastructure cleanup:

  • mlflow/models/docker_utils.py: Removed pip install virtualenv, renamed SETUP_PYENV_AND_VIRTUALENVSETUP_PYENV
  • Removed RUN pip install virtualenv from 5 Dockerfile test fixtures
  • Removed virtualenv installation from .github/actions/setup-pyenv/action.yml

Documentation:

  • CLI help text updated to clarify env_manager="virtualenv" uses venv internally
  • Model documentation updated to reflect venv usage instead of virtualenv package

The only virtualenv-specific feature used was --python to target a different interpreter, which venv achieves by invoking it directly from the target Python binary.

How is this PR tested?

  • Existing unit/integration tests
  • New unit/integration tests
  • Manual tests

Manual verification:

  • Validated removed functions no longer accessible
  • Verified env_manager constants and imports functional
  • Confirmed no remaining pip install virtualenv references

Does this PR require documentation update?

  • No. You can skip the rest of this section.
  • Yes. I've updated:
    • Examples
    • API references
    • Instructions

Does this PR require updating the MLflow Skills repository?

  • No. You can skip the rest of this section.
  • Yes. Please link the corresponding PR or explain how you plan to update it.

Release Notes

Is this a user-facing change?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release notes for MLflow users.

The env_manager="virtualenv" option now uses Python's built-in venv module instead of the third-party virtualenv package. Functionality remains unchanged.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

  • area/tracking: Tracking Service, tracking client APIs, autologging
  • area/models: MLmodel format, model serialization/deserialization, flavors
  • area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
  • area/scoring: MLflow Model server, model deployment tools, Spark UDFs
  • area/evaluation: MLflow model evaluation features, evaluation metrics, and evaluation workflows
  • area/gateway: MLflow AI Gateway client APIs, server, and third-party integrations
  • area/prompts: MLflow prompt engineering features, prompt templates, and prompt management
  • area/tracing: MLflow Tracing features, tracing APIs, and LLM tracing functionality
  • area/projects: MLproject format, project running backends
  • area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
  • area/build: Build and test infrastructure for MLflow
  • area/docs: MLflow documentation pages

How should the PR be classified in the release notes? Choose one:

  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
  • rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
  • rn/feature - A new user-facing feature worth mentioning in the release notes
  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
  • rn/documentation - A user-facing documentation change worth mentioning in the release notes

Should this PR be included in the next patch release?

  • Yes (this PR will be cherry-picked and included in the next patch release)
  • No (this PR will be included in the next minor release)
Original prompt

Replace virtualenv with python -m venv in the virtualenv env_manager path

Created by Claude Code for handoff.

Patch release: No

Summary

MLflow uses the third-party virtualenv package to create Python virtual environments when env_manager="virtualenv" is specified. The only virtualenv-specific feature used is the --python flag (to target a different Python interpreter than the one running the command). This can be replaced with python -m venv from the standard library by invoking venv directly from the target Python binary, allowing the virtualenv dependency to be removed entirely.

Note: The default env_manager is already "uv" — the virtualenv path is the legacy/fallback option. This change only affects the env_manager="virtualenv" code path.

Codebase Analysis

Core change — environment creation command

  • mlflow/utils/virtualenv.py:268-280 — The env_manager == em.VIRTUALENV branch in _create_virtualenv():
    # Current: uses sys.executable to run virtualenv, --python to target a different interpreter
    env_creation_cmd = [sys.executable, "-m", "virtualenv", "--python", python_bin_path, env_dir]
    
    # New: invoke venv directly from the target Python
    env_creation_cmd = [python_bin_path, "-m", "venv", env_dir]
    This is the only place where virtualenv is actually invoked as a tool.

Validation functions to update

  • mlflow/utils/virtualenv.py:68-72_is_virtualenv_available(): Checks shutil.which("virtualenv"). Can be removed or replaced with a check that python_bin_path -m venv works (or just removed since venv is part of stdlib).
  • mlflow/utils/virtualenv.py:75-84_validate_virtualenv_is_available(): Error message tells users to pip install virtualenv. Should be updated or removed. Consider keeping a validation that checks the target Python has venv module available (some Debian/Ubuntu system Pythons don't ship it without python3-venv apt package). This only matters for system Python — pyenv-built Python always includes venv.
  • mlflow/utils/virtualenv.py:419-421_get_or_create_virtualenv() calls both _validate_pyenv_is_available() and _validate_virtualenv_is_available() when env_manager == "virtualenv". Remove the virtualenv validation call.

Dependency removal

  • pyproject.toml:84"virtualenv" is listed in the sagemaker optional extras group. Remove it.
  • dev/pyproject.py:340"virtualenv" is listed as a dev dependency. Remove it.

Docker image update

  • mlflow/models/docker_utils.py:18-39 — The SETUP_PYENV_AND_VIRTUALENV string contains RUN pip install virtualenv (line 38). Remove that line. Consider renaming the constant to SETUP_PYENV since virtualenv is no longer installed.

Files that import/reference virtualenv but DON'T need code changes

These files use function names like _get_virtualenv_name, _get_virtualenv_activate_cmd, etc. — these are internal function names that refer to the broader concept of "virtual environments" in MLflow, not the virtualenv package specifically. Renaming them is optional and not required for correctness:

  • mlflow/pyfunc/__init__.py — Imports _get_virtualenv_name, uses EnvType.VIRTUALENV (this is a config key in MLmodel files, not the package name)
  • mlflow/pyfunc/backend.py — Imports _get_or_create_virtualenv
  • mlflow/models/container/__init__.py — Imports _get_or_create_virtualenv
  • mlflow/projects/backend/local.py — Imports multiple virtualenv utility functions
  • mlflow/server/jobs/utils.py — Imports virtualenv utility functions (but only uses the uv path, no change needed)
  • mlflow/utils/env_manager.py:6VIRTUALENV = "virtualenv" constant. This is a user-facing env_manager name, not the package. No change needed.
  • mlflow/utils/cli_args.py — Help text references "virtualenv" as an env_manager option. Update docs to clarify it uses venv internally.

Tests to update

  • tests/pyfunc/test_virtualenv.py:26-27 — Skip condition checks _is_virtualenv_available(). Update to no longer require virtualenv binary.
  • tests/pyfunc/test_virtualenv.py:18-20 — Imports _is_virtualenv_available. Update import if function is removed.
  • tests/projects/test_virtualenv_projects.py — Uses _create_virtualenv spy. No functional change needed but verify tests pass.

Success Criteria

  • _create_virtualenv() uses [python_bin_path, "-m", "venv", env_dir] instead of [sys.executable, "-m", "virtualenv", "--python", python_bin_path, env_dir] when env_manager="virtualenv"
  • _is_virtualenv_available() and _validate_virtualenv_is_available() are removed or replaced
  • virtualenv dependency removed from pyproject.toml and dev/pyproject.py
  • Docker setup in docker_utils.py no longer installs virtualenv via pip
  • CLI help text in cli_args.py updated to mention venv instead of virtualenv where appropriate
  • Existing tests in tests/pyfunc/test_virtualenv.py and `...

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: harupy <17039389+harupy@users.noreply.github.com>
Copilot AI changed the title [WIP] Replace virtualenv with python -m venv in env_manager Replace virtualenv with python -m venv in virtualenv env_manager path Feb 7, 2026
@harupy harupy marked this pull request as ready for review February 7, 2026 14:27
Copilot AI review requested due to automatic review settings February 7, 2026 14:27
@github-actions github-actions bot added area/models MLmodel format, model serialization/deserialization, flavors area/projects MLproject format, project running backends area/scoring MLflow Model server, model deployment tools, Spark UDFs rn/feature Mention under Features in Changelogs. labels Feb 7, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes the third-party virtualenv dependency by switching the legacy env_manager="virtualenv" path to use Python’s built-in venv module (invoked from the target Python interpreter), with corresponding dependency and tooling cleanup across the repo.

Changes:

  • Core: Update virtual environment creation to run python_bin_path -m venv instead of python -m virtualenv --python ...
  • Cleanup: Remove virtualenv from extras/dev dependencies and docker setup steps
  • UX/tests: Adjust CLI help text and relax a test skip condition that previously required virtualenv

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
mlflow/utils/virtualenv.py Switch env creation command to stdlib venv and remove virtualenv availability validation hook.
mlflow/utils/cli_args.py Update CLI help text and conda warning copy to reference uv / venv.
mlflow/models/docker_utils.py Remove pip install virtualenv and rename setup constant accordingly.
pyproject.toml Drop virtualenv from optional extras.
pyproject.release.toml Drop virtualenv from release extras.
libs/skinny/pyproject.toml Drop virtualenv from skinny package extras.
dev/pyproject.py Remove virtualenv from generated dev dependencies.
tests/pyfunc/test_virtualenv.py Remove virtualenv availability gating from the test skip condition.
uv.lock Lockfile update removing virtualenv from resolved extras/deps.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 7, 2026

Documentation preview for 92ff1ec is available at:

Changed Pages (1)

More info
  • Ignore this comment if this PR does not change the documentation.
  • The preview is updated when a new commit is pushed to this PR.
  • This comment was created by this workflow run.
  • The documentation was built by this workflow run.

@harupy
Copy link
Member

harupy commented Feb 7, 2026

@copilot CI is failing because the reference Dockerfile fixtures in tests/resources/dockerfile/ still contain the RUN pip install virtualenv line. Please remove that line from all Dockerfile fixtures in that directory that have it, including at least:

  • tests/resources/dockerfile/Dockerfile_sagemaker_virtualenv_no_java
  • tests/resources/dockerfile/Dockerfile_java_flavor

Also check for any other Dockerfile fixtures in tests/resources/dockerfile/ that reference RUN pip install virtualenv and update them as well. The constant was renamed from SETUP_PYENV_AND_VIRTUALENV to SETUP_PYENV so also verify there are no other references to the old name.

Co-authored-by: harupy <17039389+harupy@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 7, 2026

@copilot CI is failing because the reference Dockerfile fixtures in tests/resources/dockerfile/ still contain the RUN pip install virtualenv line. Please remove that line from all Dockerfile fixtu...

Fixed in 57c8b68. Removed RUN pip install virtualenv from all 5 Dockerfile fixtures:

  • Dockerfile_install_mlflow_virtualenv
  • Dockerfile_java_flavor
  • Dockerfile_no_model_uri
  • Dockerfile_sagemaker_virtualenv
  • Dockerfile_sagemaker_virtualenv_no_java

No references to the old SETUP_PYENV_AND_VIRTUALENV constant name were found in the test fixtures.

Co-authored-by: harupy <17039389+harupy@users.noreply.github.com>
Copilot AI requested a review from harupy February 7, 2026 16:16
@harupy
Copy link
Member

harupy commented Feb 7, 2026

@copilot remove Changes: section in the pr body

Copy link
Contributor Author

Copilot AI commented Feb 7, 2026

@copilot remove Changes: section in the pr body

I don't have the ability to edit the PR description directly. The PR description is set when the PR is created and can only be edited through the GitHub web interface or CLI. You'll need to manually remove the Changes: section from the PR description on GitHub.

@harupy
Copy link
Member

harupy commented Feb 7, 2026

@copilot clean up virtualenv in the setup-pyenv action

Co-authored-by: harupy <17039389+harupy@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 7, 2026

@copilot clean up virtualenv in the setup-pyenv action

Fixed in 6055c5f. Removed the "Install virtualenv" step from .github/actions/setup-pyenv/action.yml.

@harupy
Copy link
Member

harupy commented Feb 7, 2026

@copilot Search virtualenv in the docs and update

Co-authored-by: harupy <17039389+harupy@users.noreply.github.com>
@github-actions github-actions bot added area/build Build and test infrastructure for MLflow area/docs Documentation issues labels Feb 8, 2026
@harupy harupy added this pull request to the merge queue Feb 8, 2026
Merged via the queue into master with commit 2a3b04f Feb 8, 2026
46 checks passed
@harupy harupy deleted the copilot/replace-virtualenv-with-venv branch February 8, 2026 07:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/build Build and test infrastructure for MLflow area/docs Documentation issues area/models MLmodel format, model serialization/deserialization, flavors area/projects MLproject format, project running backends area/scoring MLflow Model server, model deployment tools, Spark UDFs rn/feature Mention under Features in Changelogs.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants