Skip to content

Frame out the env spec cep#50

Open
ericdill wants to merge 5 commits intoconda:mainfrom
ericdill:env-spec
Open

Frame out the env spec cep#50
ericdill wants to merge 5 commits intoconda:mainfrom
ericdill:env-spec

Conversation

@ericdill
Copy link
Copy Markdown

discussed with @mariusvniekerk, @AlbertDeFusco , @jezdez and @jaimergp.

opening this PR as a draft PR for further conversation. it's not really ready for full review yet.

there's a notebook that validates the env yamls that are part of this PR. those env yamls (and the validation code) should probably move over to the conda specs repo once this is finished and accepted.

dependencies:
- python=3.7
- codecov
prefix: /Users/username/anaconda3/envs/env-name
Copy link
Copy Markdown
Author

@ericdill ericdill May 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jezdez I've been poking at this name vs prefix locally with conda env create and mamba env create. Neither seem to do anything with this prefix key, even though it shows up in conda env export. It seems as though this is a valid key for the spec that conda env will accept, but conda env doesn't actually do anything with this key. can you confirm that I'm understanding this behavior correctly?

$ conda env export

name: base
channels:
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - brotlipy=0.7.0=py310h5764c6d_1005
  - bzip2=1.0.8=h7f98852_4
  - c-ares=1.18.1=h7f98852_0
  - ca-certificates=2022.12.7=ha878542_0
  - certifi=2022.12.7=pyhd8ed1ab_0
  - cffi=1.15.1=py310h255011f_3
  - charset-normalizer=3.1.0=pyhd8ed1ab_0
  - colorama=0.4.6=pyhd8ed1ab_0
  - conda=23.1.0=py310hff52083_0
  - conda-package-handling=2.0.2=pyh38be061_0
  - conda-package-streaming=0.7.0=pyhd8ed1ab_1
  - cryptography=40.0.1=py310h34c0648_0
  - fmt=9.1.0=h924138e_0
  - icu=72.1=hcb278e6_0
  - idna=3.4=pyhd8ed1ab_0
  - keyutils=1.6.1=h166bdaf_0
  - krb5=1.20.1=h81ceb04_0
  - ld_impl_linux-64=2.40=h41732ed_0
  - libarchive=3.6.2=h3d51595_0
  - libcurl=7.88.1=hdc1c0ab_1
  - libedit=3.1.20191231=he28a2e2_2
  - libev=4.33=h516909a_1
  - libffi=3.4.2=h7f98852_5
  - libgcc-ng=12.2.0=h65d4601_19
  - libgomp=12.2.0=h65d4601_19
  - libiconv=1.17=h166bdaf_0
  - libmamba=1.4.1=hcea66bb_0
  - libmambapy=1.4.1=py310h1428755_0
  - libnghttp2=1.52.0=h61bc06f_0
  - libnsl=2.0.0=h7f98852_0
  - libsolv=0.7.23=h3eb15da_0
  - libsqlite=3.40.0=h753d276_0
  - libssh2=1.10.0=hf14f497_3
  - libstdcxx-ng=12.2.0=h46fd767_19
  - libuuid=2.38.1=h0b41bf4_0
  - libxml2=2.10.3=hfdac1af_6
  - libzlib=1.2.13=h166bdaf_4
  - lz4-c=1.9.4=hcb278e6_0
  - lzo=2.10=h516909a_1000
  - mamba=1.4.1=py310h51d5547_0
  - ncurses=6.3=h27087fc_1
  - openssl=3.1.0=h0b41bf4_0
  - pip=23.0.1=pyhd8ed1ab_0
  - pluggy=1.0.0=pyhd8ed1ab_5
  - pybind11-abi=4=hd8ed1ab_3
  - pycosat=0.6.4=py310h5764c6d_1
  - pycparser=2.21=pyhd8ed1ab_0
  - pyopenssl=23.1.1=pyhd8ed1ab_0
  - pysocks=1.7.1=pyha2e5f31_6
  - python=3.10.10=he550d4f_0_cpython
  - python_abi=3.10=3_cp310
  - readline=8.2=h8228510_1
  - reproc=14.2.4=h0b41bf4_0
  - reproc-cpp=14.2.4=hcb278e6_0
  - requests=2.28.2=pyhd8ed1ab_1
  - ruamel.yaml=0.17.21=py310h1fa729e_3
  - ruamel.yaml.clib=0.2.7=py310h1fa729e_1
  - setuptools=65.6.3=pyhd8ed1ab_0
  - tk=8.6.12=h27826a3_0
  - toolz=0.12.0=pyhd8ed1ab_0
  - tqdm=4.65.0=pyhd8ed1ab_1
  - tzdata=2023c=h71feb2d_0
  - urllib3=1.26.15=pyhd8ed1ab_0
  - wheel=0.40.0=pyhd8ed1ab_0
  - xz=5.2.6=h166bdaf_0
  - yaml-cpp=0.7.0=h27087fc_2
  - zstandard=0.19.0=py310hdeb6495_1
  - zstd=1.5.2=h3eb15da_6
prefix: /home/ericdill/mambaforge

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. I've been confused by it since it started appearing.

"type": "array",
"items": {
"anyOf": [
{"type": "string"},
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How far down the MatchSpec rabbit hole do we want to go with this jsonschema? jsonschema does support various formatting validators like regex, but I'm a little hesitant to try and capture MatchSpec in a regex... Thoughts?

@mariusvniekerk @AlbertDeFusco @jezdez @wolfv @jaimergp

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels we need a JSONSchema for the MatchSpec first, and then we can refer to it if anything. Can we leave a "type hint" for now? e.g. MatchSpecStr?

<tr><td> Author(s) </td>
<td>
Eric Dill
Marius van Niekerk
Copy link
Copy Markdown
Author

@ericdill ericdill May 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mariusvniekerk you interested in being a co-author of this?

@AlbertDeFusco
Copy link
Copy Markdown

Should this CEP also mention which of the keys are strictly required and which are optional? I've found that only dependencies is required and the rest can be left off.

@jaimergp
Copy link
Copy Markdown
Contributor

For reference(s), this epic lists a few issues where similar conversations happened: conda/conda#11341

@jaimergp
Copy link
Copy Markdown
Contributor

Btw, thanks to @AlbertDeFusco I added a pydantic model to the ongoing effort on conda/schemas:

https://github.com/jaimergp/schemas/blob/revamp/conda_models/environment_file.py

@ericdill ericdill marked this pull request as ready for review May 25, 2023 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants