Skip to content

Clearer output from micromamba search#2782

Merged
jonashaag merged 14 commits intomamba-org:mainfrom
delsner:feature/micromamba-search-format
Aug 31, 2023
Merged

Clearer output from micromamba search#2782
jonashaag merged 14 commits intomamba-org:mainfrom
delsner:feature/micromamba-search-format

Conversation

@delsner
Copy link
Contributor

@delsner delsner commented Aug 24, 2023 •
edited
Loading

This PR addresses #1965 and adds clearer output to the micromamba repoquery (or search) endpoints.

The changes comprise:

  • A more compact list view from search or repoquery that only contains each version once (+ information about additional builds)
  • Additional column Subdir (as a proxy for platform/arch) in list view
  • Redirect to single package view ("pretty view") if one uses search <package-name> without a wildcard matcher (*) -> more aligned with what other package managers do
  • New "Other Versions" section in single package view

Old output

$ micromamba search boost

 Name  Version Build           Channel
───────────────────────────────────────────
 boost 1.82.0  py39hd08ceb0_0  conda-forge
 boost 1.82.0  py38h3926e13_0  conda-forge
 boost 1.82.0  py311h5c64717_0 conda-forge
 boost 1.82.0  py310h629746b_0 conda-forge
 ...
 boost 1.74.0  py39h776e71f_5  conda-forge
 boost 1.74.0  py39h776e71f_4  conda-forge
 boost 1.74.0  py39h6f7cd10_3  conda-forge
 boost 1.74.0  py39h6f7cd10_2  conda-forge
 boost 1.74.0  py38h3578e97_3  conda-forge
 boost 1.74.0  py38h3578e97_2  conda-forge
 boost 1.74.0  py38h1e2c3d9_5  conda-forge
 boost 1.74.0  py38h1e2c3d9_4  conda-forge
 boost 1.74.0  py310hd0bb7a8_5 conda-forge
 boost 1.74.0  py310hd0bb7a8_4 conda-forge
$ micromamba search '*boost'

 Name                  Version Build               Channel
───────────────────────────────────────────────────────────────
 autoboost             20.8.1  pyhd8ed1ab_0        conda-forge
 boost                 1.82.0  py310h629746b_0     conda-forge
 boost                 1.82.0  py39hd08ceb0_0      conda-forge
 boost                 1.82.0  py38h3926e13_0      conda-forge
 boost                 1.82.0  py311h5c64717_0     conda-forge
 boost                 1.80.0  py39h99de9ae_4      conda-forge
 boost                 1.80.0  py39h99de9ae_3      conda-forge
 boost                 1.80.0  py39h99de9ae_2      conda-forge
 boost                 1.80.0  py39h70e7ac2_1      conda-forge
 boost                 1.80.0  py39h4aae847_0      conda-forge
 boost                 1.80.0  py38he0a633c_3      conda-forge
 ...
 xgboost               1.5.0   py38hd68009d_1      conda-forge
 xgboost               1.5.0   py38hd68009d_0      conda-forge
 xgboost               1.5.0   py310hab06bbd_1     conda-forge
 xgboost               1.5.0   py39ha480839_0      conda-forge
 xgboost               1.5.0   py39ha480839_1      conda-forge

New output

$ micromamba search boost

boost 1.82.0 py39hd08ceb0_0 (+ 3 builds)
----------------------------------------

 File Name       boost-1.82.0-py39hd08ceb0_0.conda
 Name            boost
 Version         1.82.0
 Build           py39hd08ceb0_0
 Build Number    0
 Size            298 Kb
 License         BSL-1.0
 Subdir          osx-arm64
 URL             https://conda.anaconda.org/conda-forge/osx-arm64/boost-1.82.0-py39hd08ceb0_0.conda
 MD5             3c6b8961ecba61716cd4b636539b1816
 SHA256          8f4aa8d1a528f2efa949cf9e7667879bc0cfea8ce66bd72558c05919a87564bb

 Dependencies:
  - python_abi 3.9.* *_cp39
  - python >=3.9,<3.10.0a0
  - python >=3.9,<3.10.0a0 *_cpython
  - libcxx >=14.0.6
  - numpy >=1.21.6,<2.0a0
  - boost-cpp 1.82.0.*

 Other Versions (6):

   Version Build
  ───────────────────────────────────────
   1.80.0  py39h99de9ae_4  (+ 17 builds)
   1.78.0  py310h4566fd1_0 (+ 17 builds)
   1.77.0  py39h776e71f_0  (+  2 builds)
   1.76.0  py39h776e71f_1  (+  4 builds)
   1.75.0  py39h776e71f_0  (+  3 builds)
   1.74.0  py39h776e71f_5  (+  9 builds)
$ micromamba search '*boost'

  Name                  Version Build                             Channel     Subdir
───────────────────────────────────────────────────────────────────────────────────────
 autoboost             20.8.1  pyhd8ed1ab_0                      conda-forge noarch
 boost                 1.82.0  py310h629746b_0     (+  3 builds) conda-forge osx-arm64
 boost                 1.80.0  py39h99de9ae_4      (+ 17 builds) conda-forge osx-arm64
 boost                 1.78.0  py310h629746b_1     (+ 17 builds) conda-forge osx-arm64
 boost                 1.77.0  py39h776e71f_0      (+  2 builds) conda-forge osx-arm64
 boost                 1.76.0  py39h776e71f_1      (+  4 builds) conda-forge osx-arm64
 boost                 1.75.0  py39h776e71f_0      (+  3 builds) conda-forge osx-arm64
 boost                 1.74.0  py39h776e71f_5      (+  9 builds) conda-forge osx-arm64
 ...
 xgboost               1.7.6   cpu_py39hc10f3ae_0  (+ 11 builds) conda-forge osx-arm64
 xgboost               1.7.4   cpu_py38hfea8814_0  (+ 19 builds) conda-forge osx-arm64
 xgboost               1.7.1   cpu_py310h6b766a5_0 (+  3 builds) conda-forge osx-arm64
 xgboost               1.7.0   cpu_py310h6b766a5_0 (+  3 builds) conda-forge osx-arm64
 xgboost               1.6.2   cpu_py39h2fb42d7_2  (+  9 builds) conda-forge osx-arm64
 xgboost               1.6.1   cpu_py310he362541_0 (+  5 builds) conda-forge osx-arm64
 xgboost               1.5.1   cpu_py39h5674d11_1  (+  8 builds) conda-forge osx-arm64
 xgboost               1.5.0   py39ha480839_1      (+  4 builds) conda-forge osx-arm64

@jonashaag
Copy link
Contributor

jonashaag commented Aug 28, 2023

Suggestions for output improvement

pretty

boost 1.82.0 py39hd08ceb0_0 (+ 3 builds)
----------------------------------------

 File Name       boost-1.82.0-py39hd08ceb0_0.conda
 Name            boost
 Version         1.82.0
 Build           py39hd08ceb0_0
 Build Number    0
 Size            298 Kb
 License         BSL-1.0
 Subdir          osx-arm64
 URL             https://conda.anaconda.org/conda-forge/osx-arm64/boost-1.82.0-py39hd08ceb0_0.conda
 MD5             3c6b8961ecba61716cd4b636539b1816
 SHA256          8f4aa8d1a528f2efa949cf9e7667879bc0cfea8ce66bd72558c05919a87564bb

->

boost 1.82.0 py39hd08ceb0_0 (+ 3 builds)
────────────────────────────────────────
  Name            boost
  Version         1.82.0
  Build           py39hd08ceb0_0
  Size            298 Kb
  License         BSL-1.0
  Subdir          osx-arm64
  File Name       boost-1.82.0-py39hd08ceb0_0.conda
  URL             https://conda.anaconda.org/conda-forge/osx-arm64/boost-1.82.0-py39hd08ceb0_0.conda
  MD5             3c6b8961ecba61716cd4b636539b1816
  SHA256          8f4aa8d1a528f2efa949cf9e7667879bc0cfea8ce66bd72558c05919a87564bb

Notes:

  • _ -> ─
  • Dropped build no
  • 2 leading spaces to be consistent with the table
  • 298 Kb technically this should be kB or KiB, right?

table

$ micromamba search '*boost'

  Name                  Version Build                             Channel     Subdir
───────────────────────────────────────────────────────────────────────────────────────
 autoboost             20.8.1  pyhd8ed1ab_0                      conda-forge noarch

The columns seem misaligned, is that a copy paste problem?

jonashaag
jonashaag approved these changes Aug 28, 2023
View reviewed changes
Copy link
Contributor

@jonashaag jonashaag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool stuff!

@delsner
Copy link
Contributor Author

delsner commented Aug 29, 2023 •
edited
Loading

I added your suggestions to the PR and yes, the columns are misaligned due to copy-paste. Here's the output:

$ micromamba search '*boost'
 Name                  Version Build                             Channel     Subdir
───────────────────────────────────────────────────────────────────────────────────────
 autoboost             20.8.1  pyhd8ed1ab_0                      conda-forge noarch
 boost                 1.82.0  py310h629746b_0     (+  3 builds) conda-forge osx-arm64
 boost                 1.80.0  py38h62d72d6_0      (+ 17 builds) conda-forge osx-arm64
 boost                 1.78.0  py310h629746b_1     (+ 17 builds) conda-forge osx-arm64
 boost                 1.77.0  py39h776e71f_0      (+  2 builds) conda-forge osx-arm64
 boost                 1.76.0  py39h776e71f_1      (+  4 builds) conda-forge osx-arm64
 ...

@delsner delsner marked this pull request as ready for review August 29, 2023 16:50
@jonashaag
Copy link
Contributor

jonashaag commented Aug 29, 2023

@AntoinePrv @JohanMabille @Klaim @Hind-M @wolfv @pavelzw looking forward to your feedback to the new search output design :)

Klaim
Klaim approved these changes Aug 30, 2023
View reviewed changes
Copy link
Member

@Klaim Klaim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! 👍🏽

My only pet peeve is usage of std::map instead of std::unordered_map (a hash-map) when we are not refering to objects in the map through pointers. It's more a performance concern and can be changed later so dont bother too much if just changing the type name dont work as expected.

@delsner
Copy link
Contributor Author

delsner commented Aug 30, 2023 •
edited
Loading

Thanks for the review @Klaim, I switched to an unordered_map where it made sense. In most places we need to maintain the order of the package versions/builds, hence the map.

AntoinePrv
AntoinePrv reviewed Aug 31, 2023
View reviewed changes
AntoinePrv
AntoinePrv approved these changes Aug 31, 2023
View reviewed changes
Copy link
Member

@AntoinePrv AntoinePrv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just nitpicking

@jonashaag jonashaag merged commit 7dfbf2c into mamba-org:main Aug 31, 2023
@jonashaag
Copy link
Contributor

jonashaag commented Aug 31, 2023

Oh, I wanted to enable auto merge, not merge directly... sorry.

@delsner delsner deleted the feature/micromamba-search-format branch August 31, 2023 15:58
@JohanMabille
Copy link
Member

JohanMabille commented Sep 1, 2023

No worries, happy to have it in the next release ;)

@delsner delsner mentioned this pull request Sep 5, 2023
Closed
@gabrielsimoes gabrielsimoes mentioned this pull request Sep 18, 2023
Closed
@pavelzw pavelzw mentioned this pull request Oct 4, 2023
Closed
@zydou
Copy link

zydou commented Nov 23, 2023 •
edited
Loading

The new output style is awful for AI developers. The info of build column is essential to install pytorch or tensorflow, because we need to choose the correct python version and cuda version.

The old style:

$ ./micromamba-1.5.0-1 search pytorch=1.9.1
Getting repodata from channels...

conda-forge/linux-64                                        Using cache
conda-forge/noarch                                          Using cache


 Name    Version Build                 Channel
───────────────────────────────────────────────────
 pytorch 1.9.1   cuda110py38hf84197b_3 conda-forge
 pytorch 1.9.1   cuda112py39h4e14dd4_3 conda-forge
 pytorch 1.9.1   cuda112py38h4f2a933_3 conda-forge
 pytorch 1.9.1   cuda112py37h3bec1eb_3 conda-forge
 pytorch 1.9.1   cuda112py36h36e649e_3 conda-forge
 pytorch 1.9.1   cuda111py39hb4a4491_3 conda-forge
 pytorch 1.9.1   cuda111py38h2f85826_3 conda-forge
 pytorch 1.9.1   cuda111py37he371307_3 conda-forge
 pytorch 1.9.1   cuda111py36h8a2106e_3 conda-forge
 pytorch 1.9.1   cuda110py39h5cf7045_3 conda-forge
 pytorch 1.9.1   cpu_py36ha8b20dc_3    conda-forge
 pytorch 1.9.1   cuda110py37h4a33b93_3 conda-forge
 pytorch 1.9.1   cuda110py36he570edd_3 conda-forge
 pytorch 1.9.1   cuda102py39h2fcd037_3 conda-forge
 pytorch 1.9.1   cuda102py38ha031fbe_3 conda-forge
 pytorch 1.9.1   cuda102py37h98b7ee3_3 conda-forge
 pytorch 1.9.1   cuda102py36h3d4679f_3 conda-forge
 pytorch 1.9.1   cpu_py39hc5866cc_3    conda-forge
 pytorch 1.9.1   cpu_py38h1ee18c8_3    conda-forge
 pytorch 1.9.1   cpu_py37hf3cc979_3    conda-forge

The new style:

$ ./micromamba-1.5.3-0 search pytorch=1.9.1
Getting repodata from channels...

conda-forge/linux-64                                        Using cache
conda-forge/noarch                                          Using cache


pytorch 1.9.1 cuda110py38hf84197b_3 (+ 19 builds)
_________________________________________________

  Name            pytorch
  Version         1.9.1
  Build           cuda110py38hf84197b_3
  Size            566269 kB
  License         BSD-3-Clause
  Subdir          linux-64
  File Name       pytorch-1.9.1-cuda110py38hf84197b_3.tar.bz2
  URL             https://conda.anaconda.org/conda-forge/linux-64/pytorch-1.9.1-cuda110py38hf84197b_3.tar.bz2
  MD5             40fd35936dceddb5e71accea1660da2c
  SHA256          dea51518cff6d110b7321db25195a0770ef034bdc361023989124199bb6792f6

 Run Constraints:
  - pytorch-cpu 99999999*
  - pytorch-gpu 1.9.1*

 Dependencies:
  - __glibc >=2.17
  - __glibc >=2.17,<3.0.a0
  - _openmp_mutex >=4.5
  - cffi
  - cudatoolkit 11.0|11.0.*
  - cudnn >=8.2.1.32,<9.0a0
  - future
  - libblas * *_mkl
  - libblas >=3.9.0,<4.0a0
  - libgcc-ng >=7.5.0
  - libprotobuf >=3.18.1,<3.19.0a0
  - libstdcxx-ng >=7.5.0
  - magma >=2.5.4,<2.5.5.0a0
  - mkl 2021.*
  - mkl >=2021.3.0,<2022.0a0
  - nccl >=2.11.4.1,<3.0a0
  - ninja
  - numpy >=1.18.5,<2.0a0
  - python >=3.8,<3.9.0a0
  - python_abi 3.8.* *_cp38
  - sleef >=3.5.1,<4.0a0
  - typing_extensions

My currently workaround is using jq to filter the json output:

$ ./micromamba-1.5.3-0 search --json tensorflow-gpu=2.11 | jq '.result.pkgs.[] | {build, version}'
{
  "build": "cuda112py310h0bbbad9_0",
  "version": "2.11.1"
}
{
  "build": "cuda112py38h0bbbad9_0",
  "version": "2.11.1"
}
{
  "build": "cuda112py39h0bbbad9_0",
  "version": "2.11.1"
}
{
  "build": "cuda112py310h0bbbad9_0",
  "version": "2.11.0"
}
{
  "build": "cuda112py310h0bbbad9_1",
  "version": "2.11.0"
}
{
  "build": "cuda112py38h0bbbad9_0",
  "version": "2.11.0"
}
{
  "build": "cuda112py38h0bbbad9_1",
  "version": "2.11.0"
}
{
  "build": "cuda112py39h0bbbad9_0",
  "version": "2.11.0"
}
{
  "build": "cuda112py39h0bbbad9_1",
  "version": "2.11.0"
}

@jonashaag jonashaag mentioned this pull request Nov 23, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AntoinePrv AntoinePrv AntoinePrv approved these changes

@Klaim Klaim Klaim approved these changes

+1 more reviewer

@jonashaag jonashaag jonashaag approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@delsner @jonashaag @JohanMabille @zydou @Klaim @AntoinePrv