Replace virtualenv with python -m venv in virtualenv env_manager path#20640
Replace virtualenv with python -m venv in virtualenv env_manager path#20640
virtualenv with python -m venv in virtualenv env_manager path#20640Conversation
Co-authored-by: harupy <17039389+harupy@users.noreply.github.com>
virtualenv with python -m venv in virtualenv env_manager path
There was a problem hiding this comment.
Pull request overview
This PR removes the third-party virtualenv dependency by switching the legacy env_manager="virtualenv" path to use Python’s built-in venv module (invoked from the target Python interpreter), with corresponding dependency and tooling cleanup across the repo.
Changes:
- Core: Update virtual environment creation to run
python_bin_path -m venvinstead ofpython -m virtualenv --python ... - Cleanup: Remove
virtualenvfrom extras/dev dependencies and docker setup steps - UX/tests: Adjust CLI help text and relax a test skip condition that previously required
virtualenv
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
mlflow/utils/virtualenv.py |
Switch env creation command to stdlib venv and remove virtualenv availability validation hook. |
mlflow/utils/cli_args.py |
Update CLI help text and conda warning copy to reference uv / venv. |
mlflow/models/docker_utils.py |
Remove pip install virtualenv and rename setup constant accordingly. |
pyproject.toml |
Drop virtualenv from optional extras. |
pyproject.release.toml |
Drop virtualenv from release extras. |
libs/skinny/pyproject.toml |
Drop virtualenv from skinny package extras. |
dev/pyproject.py |
Remove virtualenv from generated dev dependencies. |
tests/pyfunc/test_virtualenv.py |
Remove virtualenv availability gating from the test skip condition. |
uv.lock |
Lockfile update removing virtualenv from resolved extras/deps. |
|
Documentation preview for 92ff1ec is available at: Changed Pages (1)
More info
|
|
@copilot CI is failing because the reference Dockerfile fixtures in
Also check for any other Dockerfile fixtures in |
Co-authored-by: harupy <17039389+harupy@users.noreply.github.com>
Fixed in 57c8b68. Removed
No references to the old |
Co-authored-by: harupy <17039389+harupy@users.noreply.github.com>
|
@copilot remove |
I don't have the ability to edit the PR description directly. The PR description is set when the PR is created and can only be edited through the GitHub web interface or CLI. You'll need to manually remove the |
|
@copilot clean up virtualenv in the setup-pyenv action |
Co-authored-by: harupy <17039389+harupy@users.noreply.github.com>
Fixed in 6055c5f. Removed the "Install virtualenv" step from |
|
@copilot Search virtualenv in the docs and update |
Co-authored-by: harupy <17039389+harupy@users.noreply.github.com>
Related Issues/PRs
N/A
What changes are proposed in this pull request?
Replaces the third-party
virtualenvpackage with Python's built-invenvmodule for theenv_manager="virtualenv"code path. The user-facing option name remains"virtualenv"for backward compatibility.Core changes:
mlflow/utils/virtualenv.py: Changed environment creation fromsys.executable -m virtualenv --python <target>to<target> -m venv_is_virtualenv_available()and_validate_virtualenv_is_available()validation functionsvirtualenvfrompyproject.toml(sagemaker extras) anddev/pyproject.pyInfrastructure cleanup:
mlflow/models/docker_utils.py: Removedpip install virtualenv, renamedSETUP_PYENV_AND_VIRTUALENV→SETUP_PYENVRUN pip install virtualenvfrom 5 Dockerfile test fixtures.github/actions/setup-pyenv/action.ymlDocumentation:
env_manager="virtualenv"uses venv internallyThe only virtualenv-specific feature used was
--pythonto target a different interpreter, which venv achieves by invoking it directly from the target Python binary.How is this PR tested?
Manual verification:
pip install virtualenvreferencesDoes this PR require documentation update?
Does this PR require updating the MLflow Skills repository?
Release Notes
Is this a user-facing change?
The
env_manager="virtualenv"option now uses Python's built-invenvmodule instead of the third-partyvirtualenvpackage. Functionality remains unchanged.What component(s), interfaces, languages, and integrations does this PR affect?
Components
area/tracking: Tracking Service, tracking client APIs, autologgingarea/models: MLmodel format, model serialization/deserialization, flavorsarea/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registryarea/scoring: MLflow Model server, model deployment tools, Spark UDFsarea/evaluation: MLflow model evaluation features, evaluation metrics, and evaluation workflowsarea/gateway: MLflow AI Gateway client APIs, server, and third-party integrationsarea/prompts: MLflow prompt engineering features, prompt templates, and prompt managementarea/tracing: MLflow Tracing features, tracing APIs, and LLM tracing functionalityarea/projects: MLproject format, project running backendsarea/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev serverarea/build: Build and test infrastructure for MLflowarea/docs: MLflow documentation pagesHow should the PR be classified in the release notes? Choose one:
rn/none- No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" sectionrn/breaking-change- The PR will be mentioned in the "Breaking Changes" sectionrn/feature- A new user-facing feature worth mentioning in the release notesrn/bug-fix- A user-facing bug fix worth mentioning in the release notesrn/documentation- A user-facing documentation change worth mentioning in the release notesShould this PR be included in the next patch release?
Original prompt
Replace
virtualenvwithpython -m venvin thevirtualenvenv_manager pathCreated by Claude Code for handoff.
Patch release: No
Summary
MLflow uses the third-party
virtualenvpackage to create Python virtual environments whenenv_manager="virtualenv"is specified. The only virtualenv-specific feature used is the--pythonflag (to target a different Python interpreter than the one running the command). This can be replaced withpython -m venvfrom the standard library by invoking venv directly from the target Python binary, allowing thevirtualenvdependency to be removed entirely.Note: The default env_manager is already
"uv"— thevirtualenvpath is the legacy/fallback option. This change only affects theenv_manager="virtualenv"code path.Codebase Analysis
Core change — environment creation command
mlflow/utils/virtualenv.py:268-280— Theenv_manager == em.VIRTUALENVbranch in_create_virtualenv():virtualenvis actually invoked as a tool.Validation functions to update
mlflow/utils/virtualenv.py:68-72—_is_virtualenv_available(): Checksshutil.which("virtualenv"). Can be removed or replaced with a check thatpython_bin_path -m venvworks (or just removed since venv is part of stdlib).mlflow/utils/virtualenv.py:75-84—_validate_virtualenv_is_available(): Error message tells users topip install virtualenv. Should be updated or removed. Consider keeping a validation that checks the target Python hasvenvmodule available (some Debian/Ubuntu system Pythons don't ship it withoutpython3-venvapt package). This only matters for system Python — pyenv-built Python always includes venv.mlflow/utils/virtualenv.py:419-421—_get_or_create_virtualenv()calls both_validate_pyenv_is_available()and_validate_virtualenv_is_available()whenenv_manager == "virtualenv". Remove the virtualenv validation call.Dependency removal
pyproject.toml:84—"virtualenv"is listed in thesagemakeroptional extras group. Remove it.dev/pyproject.py:340—"virtualenv"is listed as a dev dependency. Remove it.Docker image update
mlflow/models/docker_utils.py:18-39— TheSETUP_PYENV_AND_VIRTUALENVstring containsRUN pip install virtualenv(line 38). Remove that line. Consider renaming the constant toSETUP_PYENVsince virtualenv is no longer installed.Files that import/reference virtualenv but DON'T need code changes
These files use function names like
_get_virtualenv_name,_get_virtualenv_activate_cmd, etc. — these are internal function names that refer to the broader concept of "virtual environments" in MLflow, not thevirtualenvpackage specifically. Renaming them is optional and not required for correctness:mlflow/pyfunc/__init__.py— Imports_get_virtualenv_name, usesEnvType.VIRTUALENV(this is a config key in MLmodel files, not the package name)mlflow/pyfunc/backend.py— Imports_get_or_create_virtualenvmlflow/models/container/__init__.py— Imports_get_or_create_virtualenvmlflow/projects/backend/local.py— Imports multiple virtualenv utility functionsmlflow/server/jobs/utils.py— Imports virtualenv utility functions (but only uses theuvpath, no change needed)mlflow/utils/env_manager.py:6—VIRTUALENV = "virtualenv"constant. This is a user-facing env_manager name, not the package. No change needed.mlflow/utils/cli_args.py— Help text references "virtualenv" as an env_manager option. Update docs to clarify it usesvenvinternally.Tests to update
tests/pyfunc/test_virtualenv.py:26-27— Skip condition checks_is_virtualenv_available(). Update to no longer require virtualenv binary.tests/pyfunc/test_virtualenv.py:18-20— Imports_is_virtualenv_available. Update import if function is removed.tests/projects/test_virtualenv_projects.py— Uses_create_virtualenvspy. No functional change needed but verify tests pass.Success Criteria
_create_virtualenv()uses[python_bin_path, "-m", "venv", env_dir]instead of[sys.executable, "-m", "virtualenv", "--python", python_bin_path, env_dir]whenenv_manager="virtualenv"_is_virtualenv_available()and_validate_virtualenv_is_available()are removed or replacedvirtualenvdependency removed frompyproject.tomlanddev/pyproject.pydocker_utils.pyno longer installs virtualenv via pipcli_args.pyupdated to mention venv instead of virtualenv where appropriatetests/pyfunc/test_virtualenv.pyand `...💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.