Skip to content

TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct#16963

Merged
Cadair merged 2 commits intoastropy:mainfrom
neutrinoceros:tst/oldestdeps_w_tox_uv
Feb 6, 2025
Merged

TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct#16963
Cadair merged 2 commits intoastropy:mainfrom
neutrinoceros:tst/oldestdeps_w_tox_uv

Conversation

@neutrinoceros
Copy link
Copy Markdown
Contributor

@neutrinoceros neutrinoceros commented Sep 7, 2024

Description

Based of #16960 #16973, close #16950, close #15865.

I should highlight that, even though this goes beyond the scope initially discussed for #16950 (restricting the use of uv to the oldestdeps factor), this is actually a minimal change: setting tox to use pip OR uv internally, depending on env factors, would require additional work and I should note that I already tried and couldn't figure it out).
update: fixed it. Now an tox will continue to work without tox-uv for except for oldestdeps.

gains:

costs:

  • tox-uv is now required to run tox

It's a one-time install burden; once you have it installed, you don't need to do anything special when running tox, and it is fully transparent to anyone who relies on the test_all extra target. I should clarify that this solution doesn't require (and is not incompatible with) a system-wide install of uv*: installing tox-uv pulls uv from PyPI to your dev env.

  • By checking this box, the PR author has requested that maintainers do NOT use the "Squash and Merge" button. Maintainers should respect this when possible; however, the final decision is at the discretion of the maintainer that merges the PR.

TODO:

  • fix all CI (including circle CI)
  • adjust docs

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 7, 2024

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

@neutrinoceros neutrinoceros changed the title TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct (⏰ wait for #16960) Sep 7, 2024
@neutrinoceros neutrinoceros force-pushed the tst/oldestdeps_w_tox_uv branch from 587067a to 9a8a900 Compare September 7, 2024 04:15
@neutrinoceros neutrinoceros changed the title TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct (⏰ wait for #16960) TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct (⏰ wait for #16960 and #16973) Sep 9, 2024
@neutrinoceros neutrinoceros changed the title TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct (⏰ wait for #16960 and #16973) TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct (⏰ wait for #16960, #16973, #16975) Sep 9, 2024
@pllim pllim added this to the v7.0.0 milestone Sep 9, 2024
@neutrinoceros neutrinoceros changed the title TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct (⏰ wait for #16960, #16973, #16975) TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct (⏰ wait for #16960, #16973) Sep 10, 2024
@neutrinoceros neutrinoceros force-pushed the tst/oldestdeps_w_tox_uv branch 5 times, most recently from 00ea425 to f50d557 Compare September 10, 2024 10:01
@neutrinoceros neutrinoceros changed the title TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct (⏰ wait for #16960, #16973) TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct (⏰ wait for #16973) Sep 12, 2024
@neutrinoceros neutrinoceros force-pushed the tst/oldestdeps_w_tox_uv branch 2 times, most recently from 09b207b to d076b9c Compare September 12, 2024 14:43
@neutrinoceros neutrinoceros force-pushed the tst/oldestdeps_w_tox_uv branch 2 times, most recently from 9adc1cb to 2aa7707 Compare September 12, 2024 18:43
@neutrinoceros neutrinoceros changed the title TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct (⏰ wait for #16973) TST: have oldestdeps trully pin every direct dependency using uv's --resolution=lowest-direct Sep 12, 2024
@neutrinoceros neutrinoceros marked this pull request as ready for review September 12, 2024 18:44
Copy link
Copy Markdown
Contributor

@mhvk mhvk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure that I know enough to review, but it seems sensibe.

One suggestion: if we think regular tox should continue to work (which seems a good idea to me), we should have at least some runs in CI that use it, i.e., omit the toxdeps: tox-uv in some/most cases.

Also, if things will not work for oldestdeps, the help line for that should mention it.

Aside, I assume that something like oldestdeps: tox-uv in tox.ini is not going to be good enough, right? It would come too late, presumably.

p.s. Things like pip freeze, which is done inside tox.ini, are OK with uv?

pyproject.toml Outdated
]
dev_all = [
"tox",
"tox-uv>=1.11.3",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't tox-uv depend on tox, i.e., can we remove the line above?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, tox-uv depends on tox, but I'd rather keep all direct dependencies explicit: doing otherwise defeats the purpose of --resolution=lowest-direct (even though in this specific case it doesn't really matter). Instead, I'd like to actually give some lower bound to what version of tox we need.

@neutrinoceros
Copy link
Copy Markdown
Contributor Author

One suggestion: if we think regular tox should continue to work (which seems a good idea to me), we should have at least some runs in CI that use it, i.e., omit the toxdeps: tox-uv in some/most cases.

Yes, that was the initial plan, and I amended it because there were some hard-coded pip calls in tox.ini so it felt unescepable to hard wire uv instead, but that's not the case anymore after #16973 and #16975, so I can revert a bunch of changes here and use tox-uv only when it's needed.

Also, if things will not work for oldestdeps, the help line for that should mention it.

good call.

Aside, I assume that something like oldestdeps: tox-uv in tox.ini is not going to be good enough, right? It would come too late, presumably.

yeah unfortunately it doesn't work (already tried). I also made an attempt at implementing a way to crash oldestdeps runs early if tox-uv wasn't available at runtime but I didn't find a reliable way to do that. So I guess for now I'll stick to your suggestion of better documenting it.

p.s. Things like pip freeze, which is done inside tox.ini, are OK with uv?

That was the point of #16973 and #16975: tox has native substitution mechanisms and tox-uv brings the necessary changes behind the curtain. In particular it changes the default install_command from python -I -m pip install to uv pip install and changes list_dependencies_command from pip freeze --all to uv pip freeze.

@neutrinoceros neutrinoceros force-pushed the tst/oldestdeps_w_tox_uv branch 2 times, most recently from a33f3b7 to 55ef392 Compare September 13, 2024 05:47
tox.ini Outdated
Comment on lines +28 to +29
devdeps: UV_EXTRA_INDEX_URL = https://pypi.anaconda.org/liberfa/simple https://pypi.anaconda.org/scientific-python-nightly-wheels/simple
mpldev: PIP_EXTRA_INDEX_URL = https://pypi.anaconda.org/scientific-python-nightly-wheels/simple
mpldev: UV_EXTRA_INDEX_URL = https://pypi.anaconda.org/scientific-python-nightly-wheels/simple
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need UV_EXTRA_INDEX_URL that would only affect dev jobs, not oldestdeps?

Copy link
Copy Markdown
Contributor Author

@neutrinoceros neutrinoceros Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of how tox-uv is enabled by mere installation, so, even if it doesn't happen in CI, this is needed to allow running dev deps jobs locally if tox-uv is installed.

@neutrinoceros
Copy link
Copy Markdown
Contributor Author

rebased following #17675

@neutrinoceros
Copy link
Copy Markdown
Contributor Author

Indeed the bump is causing a regression. Reported as #17681
Switching this PR to draft in the mean time.

@neutrinoceros neutrinoceros marked this pull request as draft January 28, 2025 10:27
@pllim pllim mentioned this pull request Feb 3, 2025
1 task
@neutrinoceros neutrinoceros force-pushed the tst/oldestdeps_w_tox_uv branch from 700764a to 2fde9d1 Compare February 4, 2025 07:12
@neutrinoceros
Copy link
Copy Markdown
Contributor Author

rebased to fix a conflict with #17711

@neutrinoceros neutrinoceros force-pushed the tst/oldestdeps_w_tox_uv branch from 2fde9d1 to c22fe66 Compare February 5, 2025 13:31
@neutrinoceros neutrinoceros marked this pull request as ready for review February 5, 2025 13:49
@neutrinoceros
Copy link
Copy Markdown
Contributor Author

neutrinoceros commented Feb 5, 2025

This is back to a stable state. Anyone up to bite the bullet ?

@pllim
Copy link
Copy Markdown
Member

pllim commented Feb 5, 2025

I think it is time we make a decision.

  • Pro: tox-uv will test oldestdeps with really the minversions we claim in pyproject.toml
  • Con: tox-uv becomes a dependency of any tox run, not just oldestdeps and some env var names have to be changed to UV stuff.

I see there are already 2 approvals. Should we merge soon or are there some remaining concerns that would be blockers?

Copy link
Copy Markdown
Member

@Cadair Cadair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am still +0 on this, mainly because there's no way to not use uv for some tox factors and I am not a massive fan of having substantially more limited testing using pip which is what the majority of our users will be using.

I will also point out that "use uv or not have one source of truth" isn't strictly a fair comparison as minimum_dependancies exists and works well without using uv.

tox.ini Outdated
Comment on lines +29 to +31
devdeps: UV_INDEX_STRATEGY = unsafe-best-match # match pip's behavior
mpldev: UV_INDEX = https://pypi.anaconda.org/scientific-python-nightly-wheels/simple
mpldev: UV_INDEX_STRATEGY = unsafe-best-match # match pip's behavior
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
devdeps: UV_INDEX_STRATEGY = unsafe-best-match # match pip's behavior
mpldev: UV_INDEX = https://pypi.anaconda.org/scientific-python-nightly-wheels/simple
mpldev: UV_INDEX_STRATEGY = unsafe-best-match # match pip's behavior
devdeps, mpldev: UV_INDEX_STRATEGY = unsafe-best-match # match pip's behavior
mpldev: UV_INDEX = https://pypi.anaconda.org/scientific-python-nightly-wheels/simple

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks ! done in a slightly different order so that both UV_INDEX lines sit next to each other.

@neutrinoceros neutrinoceros force-pushed the tst/oldestdeps_w_tox_uv branch from c22fe66 to 7da18bb Compare February 6, 2025 09:02
@neutrinoceros
Copy link
Copy Markdown
Contributor Author

I will also point out that "use uv or not have one source of truth" isn't strictly a fair comparison as minimum_dependencies exists and works well without using uv.

Fair point. Although this tool doesn't work out of the box for this case1, we can always fix it and switch if we encounter issues with uv/tox-uv.

Footnotes

  1. I specifically tested minimum_dependencies astropy --extras test_all and it doesn't seem to understand self-referential requirements like astropy[test_all] -> astropy[all]. It also has the drawback of needing to be installed in the same env as the package whose requirements it's reading, which adds complexity in the installation process that uv avoids by resolving dependencies before installing anything.

@Cadair
Copy link
Copy Markdown
Member

Cadair commented Feb 6, 2025

  1. it doesn't seem to understand self-referential requirements like astropy[test_all] -> astropy[all]

Yes this is a pain (OpenAstronomy/minimum_dependencies#30) but it's not the end of the world to list all the extras.

@neutrinoceros
Copy link
Copy Markdown
Contributor Author

but it's not the end of the world to list all the extras.

sure, but then you're back to not having a single source of truth 🙃

@Cadair
Copy link
Copy Markdown
Member

Cadair commented Feb 6, 2025

sure, but then you're back to not having a single source of truth 🙃

Well you have a source of truth for the deps, if not all the extras. 🤷 It's a bug which should be fixed, but it's in the deps parsing code in some other library and is generally hard to unravel.

@Cadair
Copy link
Copy Markdown
Member

Cadair commented Feb 6, 2025

I am officially bored of this conversation.

@Cadair Cadair merged commit 524675d into astropy:main Feb 6, 2025
23 checks passed
@neutrinoceros neutrinoceros deleted the tst/oldestdeps_w_tox_uv branch February 6, 2025 10:11
@mhvk
Copy link
Copy Markdown
Contributor

mhvk commented Feb 19, 2025

Would this still work?

@mhvk Thanks for pointing this out, I honestly had no idea that someone would use tox to create virtualenvs. The answer is a yes but: uv does not include pip itself in virtualenvs it creates (because it makes it redundant), so if you were to run the exact commands you showed in that order, you'd get stopped at

$ pip install ipython
/Users/clm/dev/astropy-project/coordinated/astropy/.tox/test/bin/python: No module named pip

but that's trivially solved by either:

* typing `uv pip` instead of `pip`

* running the awkward (but functional) command `uv pip install pip` between env activation and any further install

is this satisfactory ?

@neutrinoceros - now that this PR is merged, I noticed that my workflow of

tox -e test --develop --notest
pip install ipython

is indeed broken (as expected). However, your statement that I could just do uv pip install ipython turns out not to be true, since by default uv is not in the path, so I'm stuck with .tox/.tox/bin/uv pip install ipython, which is quite long. There probably is a clever trick to have this work, but I haven't been able to find it. Any suggestions?

@pllim
Copy link
Copy Markdown
Member

pllim commented Feb 19, 2025

I am sure others might run into the same problem, so let's move follow-up discussions to the issue below. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TST: Decouple alldeps from oldestdeps Pin oldestdeps in tox.ini to exact versions?

8 participants