[Tools] Improve setup_local_ce.py script RC versions support and image tag selection #8802
Conversation
| # Always use arm64 for Nuclio images regardless of provided arch | ||
| nuclio_arch = "arm64" |
| # Always use arm64 for Nuclio images regardless of provided arch | ||
| nuclio_arch = "arm64" | ||
|
|
||
| # For jupyter and log-collector images, drop a leading 'v' if provided |
There was a problem hiding this comment.
| # For jupyter and log-collector images, drop a leading 'v' if provided | |
| # For mlrun images, drop a leading 'v' if provided |
| ) | ||
| return (maj, min_, patch, is_final, rc_num), cleaned | ||
|
|
||
| candidates: list[tuple[tuple[int, int, int, int, int], str]] = [] |
There was a problem hiding this comment.
use namedtuple, it would be better
| if not candidates: | ||
| raise ValueError(f"No semver tag found for {repo_name}") | ||
|
|
||
| best_key, best_cleaned = max(candidates, key=lambda t: t[0]) |
There was a problem hiding this comment.
what do you max on? the major? if all versions are [(1,1,0), (1,2,0)] how will it take the second item?
| return tags | ||
|
|
||
|
|
||
| def get_latest_valid_version(repo_name: str) -> str: |
There was a problem hiding this comment.
I dont understand the definition of "latest valid"). I think we need to condition these 2 flows into
- by default, get the latest GA versions and use them
- if provided with
--develor so flag, take latest-greatest RC available
liranbg
left a comment
There was a problem hiding this comment.
still relevant - https://github.com/mlrun/mlrun/pull/8802/files#r2476835951
| """ | ||
| Return the *greatest semantic* release tag, optionally excluding prereleases. | ||
| """ | ||
| releases = get_all_releases(repository) |
There was a problem hiding this comment.
why do you need to get all_releases and not get the release marked as latest?
There was a problem hiding this comment.
/releases/latest only returns the most recent stable release and skips RCs or newer tags, so we fetch all releases to pick the true latest by semver.
Description
Refactors and hardens the local CE installer to improve version discovery, default image selection, and operational robustness. The script now uses
semverfor GitHub tag ordering (with optional RC support), prefers chart-pinned image tags by default, exposes explicit CLI switches for version policy, and strengthens Helm/Kubernetes flows and debug output.Changes
Version discovery and policy
semver-based parsing and ordering for tags, including normalization of prerelease labels (rc38 → rc.38) to ensure correct comparisons.VersionSourceandGitHubApiPathenums and centralized GitHub access in_github_get_json(...)with optional token auth.resolve_from_releases(...)– highest semantic release (optionally includes prereleases).resolve_from_tags(...)– tag-based fallback.choose_first_matching_version(...)– stable-only vs. stable+RC enforcement.get_latest_valid_version(...)with:allow_dev_versions: bool– enables RC selection.version_source: "auto" | "release" | "tag"– controls and orders source preference.Chart defaults vs. “latest”
_read_chart_image_versions(...)to read MLRun/Nuclio tags from chart values (cleaning leadingvand architecture suffices).setup_ce(...)/install_ce(...)now auto-resolve missing CE/MLRun/Nuclio versions using chart values or GitHub, respecting--dev-versions.CLI and UX
--dev-versions– allow RC tags.--use-latest-versions– ignore chart tags and query GitHub.pathlib.Pathand strengthened typing across helpers.echo_color(...),run_command(...), and related helpers.Command execution, Helm, and Kubernetes
run_command(...)now:--debugto Helm commands when debug mode is on.[COMMAND OUTPUT]on failures before raising/warning.ensure_helm_repo(...)andupdate_helm_repos(...).helm statusduring ingress setup./etc/hostssafety:clear_namespaces(...)skips missing namespaces instead of erroring./etc/hostsupdates now back up the file and apply changes via a temp file swap.Image upgrade flow
upgrade_images(...):use_chart_versions=True.--use-latest-versionsis set.vfrom MLRun/Jupyter tags before applying chart values.install_ce(...)orchestrates the entire workflow with the new version-policy logic and improved logging.Testing
1.10.0-rc38>1.9.2) when--dev-versionsis used.--use-latest-versionscorrectly pulls versions from GitHub with fallback logic.[COMMAND OUTPUT].Breaking Changes
https://iguazio.atlassian.net/browse/ML-11621