Skip to content

build: improve installation making use of pyproject.toml file only and setuptools#491

Merged
gcroci2 merged 68 commits intomainfrom
cleaning_installation_gcroci2
Sep 15, 2023
Merged

build: improve installation making use of pyproject.toml file only and setuptools#491
gcroci2 merged 68 commits intomainfrom
cleaning_installation_gcroci2

Conversation

@gcroci2
Copy link
Copy Markdown
Collaborator

@gcroci2 gcroci2 commented Sep 6, 2023

  • I removed redundancies from the actions
  • Now we have only one config file instead of 3 (pyproject.toml), which contains all the needed info for pip-related dependencies and metadata of deeprank2.
  • I fixed the builds for multiple python versions; the action.yml file was not taking as input the python version specified in the workflows, thus defaulting always to the one specified (as a default) in the action.yml, which was python 3.9. We were not running workflows on python 3.10.
  • Now we test the build on Python 3.10 and 3.11; I removed python 3.9 from everywhere, so we're not maintaining it anymore.
  • I kept the conda installation of msms since ResidueDepth from biopython uses it (tried and verified).
  • Apparently dependencies' urls (or index urls) are not expected from setuptools: the user should be in charge of choosing the indexes, and not the creator of the package; installing torch, torch_geometric and the other torch adds is not possible using setuptools beacause their installation needs such links (or conda, which is not supported by the toml project file). I've read also here that setuptools does not support such links, and that they don't comply to the new pep specifications. With poetry this was possible, but being not in line with the standards and having chosen to not use poetry for installing the other dependencies of this package, I think we should just instruct the user to install the pytorchs versions suitable for the machine, and we should not be responsible for that. As developers, we'll use actions/install-python-and-package/action.yml as a reference for how to install torch packages.
  • Same as above for the other external dependencies. Also, is a pain to maintain the instructions update in the readme, so I just left the links to the right installation pages for all the external packages (i.e., packages not installed through the toml file). This makes our readme instructions much more readable, and avoids us to keep updating the readme each time one external dependency changes something (see now, MSMS removed the download file from the website). Eventually, we can update the links for pointing to more up-to-date installation pages, when and if they'll change. Thus, I updated the installation docs.
  • I removed h5xplorer from everywhere. This PR closes issue Remove h5explorer from package #456 as well.
  • I tried to setup MacOS installation, and it was running correctly (at least for Python 3.10) except for MSMS installation, which for Mac can't be via conda. I opened an issue about it on their repo, in case they solve it I will readd the MacOS installation. For now I left the OS checking in the action.yml, and in case it will be ready to be run.

@gcroci2 gcroci2 self-assigned this Sep 6, 2023
@gcroci2 gcroci2 linked an issue Sep 6, 2023 that may be closed by this pull request
3 tasks
@gcroci2 gcroci2 requested a review from DaniBodor September 13, 2023 16:44
Copy link
Copy Markdown
Collaborator

@DaniBodor DaniBodor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job, big effort!

PR should be renamed as you are not using poetry in the end. Also, should it be a "build" type rather than a "refactor"?

I could follow the install instructions and could run pytest and prospector without anything getting flagged.
I did notice that somewhere along the way scipy 1.11.1 gets installed, then gets uninstalled and 1.11.2 gets installed. Probably not worthwhile investigating further though.

I left some minor comments here and there.

Comment on lines 77 to 80
git clone https://github.com/DeepRank/deeprank2
cd deeprank2
pip install -e ./
pip install -e .
```
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be nice to also include the command for adding the test packages as well (I always forget the where it needs to go and which part needs to be in quotations): pip install -e '.[test]'

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, but JFYTK you can look at the action.yml file, and there you'll see how the test dependencies are added :)

README.md Outdated
* [Here](https://ssbio.readthedocs.io/en/latest/instructions/msms.html) for MacOS with M1 chip users
* [PyTorch](https://pytorch.org/get-started/locally/)
* We support torch's CPU library as well as CUDA
* [PyG](https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html) and its optional dependencies: `torch_scatter`, `torch_sparse`, `torch_cluster`, `torch_spline_conv`
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider adding them to dependencies instead?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remembered why I didn't do it at the end. As you can see here, installing torch via pip requires an --index-url; same for pytorch geometric additional deps: https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html. Could be that we're lucky and pip picks the right things for ubuntu (even it would be weird but still could be), but I'd stick to the official docs which require urls, that can't be provided in the toml file if we're using setuptools. Also in the future it will be easier to add macos build since there pytorch and pytorch geometric can't be installed via pip.

@gcroci2 gcroci2 changed the title refactor: improve installation and make use of poetry instead of setuptools build: improve installation making use of pyproject.toml file only and setuptools Sep 14, 2023
@gcroci2
Copy link
Copy Markdown
Collaborator Author

gcroci2 commented Sep 14, 2023

Great job, big effort!

PR should be renamed as you are not using poetry in the end. Also, should it be a "build" type rather than a "refactor"?

I could follow the install instructions and could run pytest and prospector without anything getting flagged. I did notice that somewhere along the way scipy 1.11.1 gets installed, then gets uninstalled and 1.11.2 gets installed. Probably not worthwhile investigating further though.

I left some minor comments here and there.

I changed the PR name :) regarding scipy, I think it gets installed with torch and then when we install the package via the toml file it gets upgraded because of the requirement there (>= 1.11.2), which is needed. I think it's fine for now

gcroci2 and others added 5 commits September 14, 2023 16:53
Co-authored-by: Dani Bodor <d.bodor@esciencecenter.nl>
Co-authored-by: Dani Bodor <d.bodor@esciencecenter.nl>
Co-authored-by: Dani Bodor <d.bodor@esciencecenter.nl>
Co-authored-by: Dani Bodor <d.bodor@esciencecenter.nl>
@gcroci2 gcroci2 merged commit fbc545f into main Sep 15, 2023
@gcroci2 gcroci2 deleted the cleaning_installation_gcroci2 branch September 15, 2023 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cleaning installation Remove h5explorer from package

2 participants