Skip to content

Infer Python version from repository pyproject.toml for dev versions#18838

Merged
harupy merged 1 commit intomlflow:masterfrom
harupy:infer-python-version-from-repo
Nov 14, 2025
Merged

Infer Python version from repository pyproject.toml for dev versions#18838
harupy merged 1 commit intomlflow:masterfrom
harupy:infer-python-version-from-repo

Conversation

@harupy
Copy link
Member

@harupy harupy commented Nov 14, 2025

🛠 DevTools 🛠

Open in GitHub Codespaces

Install mlflow from this PR

# mlflow
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/18838/merge
# mlflow-skinny
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/18838/merge#subdirectory=libs/skinny

For Databricks, use the following command:

%sh curl -LsSf https://raw.githubusercontent.com/mlflow/mlflow/HEAD/dev/install-skinny.sh | sh -s pull/18838/merge

What changes are proposed in this pull request?

This PR enables automatic Python version inference for dev versions by fetching and parsing the requires-python field from the package repository's pyproject.toml.

Previously, dev versions would fall back to a hardcoded default Python version (3.10). Now, the script fetches the actual requires-python specification from the repository and selects the minimum compatible Python version.

Changes:

  • Added repo field to ml-package-versions.yml for 30 packages with install_dev
  • Enhanced set_matrix.py to fetch pyproject.toml from GitHub repositories
  • Added logging to stderr for debugging Python version inference

Example: sklearn dev version now correctly uses Python 3.11 (matching requires-python = ">=3.11" in their repo)

How is this PR tested?

  • Existing unit/integration tests
  • Manual tests

Manually tested with:

uv run python dev/set_matrix.py --flavors sklearn --versions dev

Verified that sklearn dev version uses Python 3.11 and langchain (monorepo) correctly fetches from subdirectory.

Does this PR require documentation update?

  • No. You can skip the rest of this section.

Release Notes

Is this a user-facing change?

  • No. You can skip the rest of this section.

How should the PR be classified in the release notes? Choose one:

  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section

Should this PR be included in the next patch release?

  • Yes (this PR will be cherry-picked and included in the next patch release)
  • No (this PR will be included in the next minor release)

🤖 Generated with Claude Code

This change enables automatic Python version inference for dev versions by fetching and parsing the requires-python field from the package repository's pyproject.toml file.

Previously, dev versions would fall back to a hardcoded default Python version (3.10). Now, the script fetches the actual requires-python specification from the repository and selects the minimum compatible Python version from the candidates.

Changes:
- Added repo field to ml-package-versions.yml for 30 packages with install_dev
- Enhanced set_matrix.py to fetch pyproject.toml from GitHub repositories
- Added logging to stderr for better debugging of Python version inference

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>
@github-actions github-actions bot added the rn/none List under Small Changes in Changelogs. label Nov 14, 2025
@harupy harupy added enable-dev-tests Enables cross-version tests for dev versions only-latest If applied, only test the latest version of each group in cross-version tests. labels Nov 14, 2025

pytorch:
package_info:
repo: https://github.com/pytorch/pytorch/tree/HEAD
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use HEAD to avoid using master/main

resp = requests.get(raw_url, timeout=10)
resp.raise_for_status()
except requests.HTTPError as e:
if e.response.status_code == 404:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pyproject.toml might not exist. Handle 404.

@github-actions
Copy link
Contributor

Documentation preview for 29fa3bc is available at:

More info
  • Ignore this comment if this PR does not change the documentation.
  • The preview is updated when a new commit is pushed to this PR.
  • This comment was created by this workflow run.
  • The documentation was built by this workflow run.



def infer_python_version(package: str, version: str) -> str:
def _requires_python_from_repo(repo_url: str) -> str | None:
Copy link
Member Author

@harupy harupy Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed the right python version is inferred (3.11 for sklearn: scikit-learn/scikit-learn#32646):

image

Copy link
Collaborator

@serena-ruan serena-ruan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@harupy harupy merged commit 4ae8654 into mlflow:master Nov 14, 2025
120 of 208 checks passed
@harupy harupy deleted the infer-python-version-from-repo branch November 14, 2025 07:03
jackiehimel pushed a commit to jackiehimel/mlflow that referenced this pull request Nov 21, 2025
…lflow#18838)

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Jackie Himel <jacqueline.himel@vanderbilt.edu>
mprahl pushed a commit to opendatahub-io/mlflow that referenced this pull request Nov 21, 2025
…lflow#18838)

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Tian-Sky-Lan pushed a commit to Tian-Sky-Lan/mlflow that referenced this pull request Nov 24, 2025
…lflow#18838)

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Tian Lan <sky.blue266000@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enable-dev-tests Enables cross-version tests for dev versions only-latest If applied, only test the latest version of each group in cross-version tests. rn/none List under Small Changes in Changelogs. team-review Trigger a team review request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants