Skip to content

DEP: add missing lower bounds to dependencies from oldestdeps-alldeps#16960

Merged
pllim merged 1 commit intoastropy:mainfrom
neutrinoceros:dep/all_lower_bounds
Sep 12, 2024
Merged

DEP: add missing lower bounds to dependencies from oldestdeps-alldeps#16960
pllim merged 1 commit intoastropy:mainfrom
neutrinoceros:dep/all_lower_bounds

Conversation

@neutrinoceros
Copy link
Copy Markdown
Contributor

@neutrinoceros neutrinoceros commented Sep 6, 2024

Description

While working on #16950, I found that the proposed solution to use uv pip install --resolution=lowest-direct in oldestdeps only really worked for dependencies with an explicit lower bound, so, after a lot of trial and errors, here's all the missing metadata I was able to discover. I think it has value beyond the scope of #16950 so I'm proposing it as its own PR. It actually end up being most of the change needed to implement "oldestdeps with uv" (the rest of the work will happen over tox.ini).

I also threw in the release date for each of these dependencies that doesn't use calendar-based-versionning.

  • By checking this box, the PR author has requested that maintainers do NOT use the "Squash and Merge" button. Maintainers should respect this when possible; however, the final decision is at the discretion of the maintainer that merges the PR.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 6, 2024

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 6, 2024

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

@pllim pllim added this to the v7.0.0 milestone Sep 6, 2024
"asdf-astropy>=0.3",
"bottleneck",
"certifi>=2022.6.15.1",
"dask[array]>=2022.5.1",
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For dask I was able to actually test this lower bound (the previous one, 2022.8.1, was just a guess I took in #16903)

pyproject.toml Outdated
"pytest>=7.3.0", # 2023-04-08
"pytest-doctestplus>=0.12", # 2022-02-25
"pytest-astropy-header>=0.2.1", # 2022-03-10
"pytest-astropy>=0.10.0", # 2022-04-12
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if some of the pytest- pins are outdated. Most of them should have come from pytest-astropy meta package.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is actually more restrictive than what's specified in pytest-astropy 0.10.0.
Furthermore, --resolution=lowest-direct doesn't take the lower bound of indirect dependencies: we should still specify some lower bound for ourselves if we really mean to check compatibility with anything older than the most recent version of these packages.

https://github.com/astropy/pytest-astropy/blob/v0.10.0/setup.cfg

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't take the lower bound of indirect dependencies

Tsk tsk. Doesn't this defeats the purpose of our pytest-astropy meta-package pins then?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, uv also supports --resolution=lowest, which is probably closer to what you expect here, but it's much less likely to work for us given how large our fully resolved dependency tree, so, I'm going for the strategy with the highest chance of success first. We can maybe explore this option later on if you'd like, but I must say I am pessimistic about its odds.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

@eerovaher eerovaher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Writing down the minimum supported versions for all the dependencies is a good thing, but I don't see what's the benefit of writing down the release dates of all these versions.

pyproject.toml Outdated
"dask[array]>=2022.5.1",
"h5py>=3.8.0", # 2023-01-23
"pyarrow>=10.0.1", # 2022-11-22
"beautifulsoup4>=4.9.3", # 2020-10-03, imposed by pandas==2.0
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is good to record that this version requirement is imposed by another package, much more valuable than the release date.

@neutrinoceros
Copy link
Copy Markdown
Contributor Author

Writing down the minimum supported versions for all the dependencies is a good thing, but I don't see what's the benefit of writing down the release dates of all these versions.

I often find myself wondering if some dependencies are old enough that maybe I can bump them without feeling guilty, so this helps with that.
But to be frank, this is mostly something I got for free since I spent most of my time working on this waiting for tests to complete in sessions of 2min so I grabbed these to keep myself busy.

@neutrinoceros
Copy link
Copy Markdown
Contributor Author

Another gain I discovered while migrating a bunch of projects' CI to uv is that having release dates right here makes it much easier if we need to set --exclude-newer in the future.

@neutrinoceros
Copy link
Copy Markdown
Contributor Author

Please let me know if the inclusion of release dates is a show stopper for this PR and I'll remove them, they're not essential.

@eerovaher
Copy link
Copy Markdown
Member

The release dates add clutter to our configuration, they bloat the patch and they create a needless implicit expectation that the version release dates must be updated together with the dependencies themselves.

I often find myself wondering if some dependencies are old enough that maybe I can bump them without feeling guilty, so this helps with that.

The date in the calendar is not a valid reason for updating our dependency requirements. Valid reasons are either security related, in which case release dates don't matter, or the removal or avoidance of version-dependent code in astropy, in which case what matters is the release date of the new oldest supported version, which is would have to be looked up from an external resource anyways.

@neutrinoceros
Copy link
Copy Markdown
Contributor Author

removed

Copy link
Copy Markdown
Member

@eerovaher eerovaher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is good to have explicit and tested version requirements for all our dependencies.

@pllim pllim merged commit 35bba8d into astropy:main Sep 12, 2024
@pllim
Copy link
Copy Markdown
Member

pllim commented Sep 12, 2024

Thanks, all!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants