Fix dask-core / distributed pinning to properly exclude nightlies#191
Fix dask-core / distributed pinning to properly exclude nightlies#191jrbourbeau merged 2 commits intoconda-forge:mainfrom
dask-core / distributed pinning to properly exclude nightlies#191Conversation
|
Hi! This is the friendly automated conda-forge-linting service. I just wanted to let you know that I linted all conda-recipes in your PR ( |
|
@conda-forge-admin, please rerender |
|
Hi! This is the friendly automated conda-forge-webservice. I tried to rerender for you, but it looks like there was nothing to do. This message was generated by GitHub actions workflow run https://github.com/conda-forge/dask-feedstock/actions/runs/2868228488. |
jrbourbeau
left a comment
There was a problem hiding this comment.
Thanks @charlesbluca! Seems reasonable to me -- @jakirkham do you have a minute to look at this?
jrbourbeau
left a comment
There was a problem hiding this comment.
Thanks @charlesbluca -- will plan to merge this at EOD if no further comments
jakirkham
left a comment
There was a problem hiding this comment.
Thanks for continuing to explore this Charles! 🙏
Am still a bit concerned by the use of .s here. Either we depending on undocumented behavior (which could then change later) or we are running into some kind of bug (which could be fixed later).
Given this, would suggest simplifying our versioning syntax by breaking things out. This should hopefully avoid deep dives into how Conda & Conda-Build are parsing version constraints (either now or in the future when addressing another issue). More concrete suggestions along these lines are shown below
recipe/meta.yaml
Outdated
| - dask-core ={{ version }},!={{ version }}a* | ||
| - distributed ={{ version }},!={{ version }}a* | ||
| - dask-core {{ version }}.*,!={{ version }}a.* | ||
| - distributed {{ version }}.*,!={{ version }}a.* |
There was a problem hiding this comment.
Am still a little worried about adding .s here not behaving correctly. From the Conda-Builds docs on run:
Package names should follow the package match specifications.
From the match spec docs:
* matches 0 or more characters in the version string. In terms of regular expressions, it is the same as r'.*'.
The fuzzy constraint numpy=1.11 matches 1.11, 1.11.0, 1.11.1, 1.11.2, 1.11.18, and so on.
So the previous syntax should already be doing what we want. The fact it isn't is a little strange and may indicate a bug.
The use of .s shouldn't be needed and may lead to incorrect behavior.
Given all this, am wondering if the , is what is causing us issues
There was a problem hiding this comment.
I think the comma could be what's causing the issue here - I noticed that from the conflict message I shared in #190 (comment) that a .* seemingly got appended to the end of our constraint automatically.
I'm not sure if this is documented anywhere, but I would assume it gets done to map fuzzy constraints from the conda install style to the conda build style, i.e.
numpy=1.11 -> numpy 1.11.*
but might not be expecting there to be two constraints in one line and so ends up behaving strangely.
I'll try out your suggestions and see if those get the intended behavior - out of interest, is there a way to inspect the dependencies of a locally built conda package? Trying mamba repoquery depends but not sure if it has support for local packages
|
Charles and I got off a call earlier where we have found a solution that works using As discussed in our call, a similar approach could also be applied to the nightlies. Though this wasn't tested yet. |
| - {{ pin_compatible('dask-core', max_pin='x.x.x') }} | ||
| - {{ pin_compatible('distributed', max_pin='x.x.x') }} |
There was a problem hiding this comment.
Using the current version, this would generate the following constraints:
- dask-core >=2022.8.0,<2022.8.1.0a0
- distributed >=2022.8.0,<2022.8.1.0a0
One edge case @jakirkham noted that this wouldn't cover is nightly packages of security backports (i.e. something like 2022.8.0.1a20220819; however, given the fact that it seems unlikely we would need to generate nightlies from backports, I think this is a reasonable solution.
|
@jrbourbeau, could you please take a look at this as well? If this seems reasonable, would like to merge it before we merge the release PR. That way the new release will have the right version constraint to start 🙂 |
As brought up in #190 (comment), the pinning introduced in #190 accidentally blocked the security backport proposal - tested this new pinning out locally and it seems to work with backports while also blocking nightly packages.
Checklist
0(if the version changed)conda-smithy(Use the phrase@conda-forge-admin, please rerenderin a comment in this PR for automated rerendering)