In #8255 which fixed #8252, we disabled pip's build isolation used when building wheels from sources with a pyproject.toml file. My understanding is that the main downside of this change is that we are now responsible for ensuring that the dependencies a project specifies in their pyproject.toml file following PEP 518 are already installed in the build environment before trying to install that package. (There are some other minor downsides such as being unable to install packages with conflicting build requirements discussed at https://www.python.org/dev/peps/pep-0517/#recommendations-for-build-frontends-non-normative, but I'm not particularly worried about any of them in the context of Certbot right now). My first question here is this understanding correct? Are there any other problems I am not aware of?
As a test of that understanding, #8255 says:
It may however be problematic if some libraries require another build system than setuptools and do not provide a fallback to a setuptools build. For the record, dns-lexicon, that I maintain, uses poetry and so a PEP 517 compliant definition of a build system, but provides also this fallback (https://github.com/AnalogJ/lexicon/blob/master/setup.py).
If dns-lexicon or another project removed their setup.py file and we couldn't use their wheel, this approach of turning off pip's build isolation should continue to work as long as we have any dependencies, such as poetry, installed in the build environment correct?
If I'm understanding things correctly, my next question is if there is future work to be done here. Should we be disabling build isolation in other locations such as the DNS plugin snaps, Dockerfiles, CI config, or our scripts in tools such as the venv*.py scripts or the release script? How will we ensure that build dependencies continue to be properly satisfied as we update the pinnings of our dependencies? Do we want to make a request or start a conversation with any of our dependencies about improvements here?
Two ideas for the last question so far are:
- Asking all of our dependencies to pin exact versions of their PEP 518 style build dependencies.
- Ask
pip for a way to have it continue to install the PEP 518 style build dependencies for us with the ability to further constrain those dependencies with a constraints file. These dependencies could be installed either in an isolated environment or the normal installation environment, although I think the former is preferred because it'd allow us to pin tools like poetry without unnecessarily adding it to our installation (or manually removing it after installing any packages that need it to build).
In #8255 which fixed #8252, we disabled pip's build isolation used when building wheels from sources with a
pyproject.tomlfile. My understanding is that the main downside of this change is that we are now responsible for ensuring that the dependencies a project specifies in theirpyproject.tomlfile following PEP 518 are already installed in the build environment before trying to install that package. (There are some other minor downsides such as being unable to install packages with conflicting build requirements discussed at https://www.python.org/dev/peps/pep-0517/#recommendations-for-build-frontends-non-normative, but I'm not particularly worried about any of them in the context of Certbot right now). My first question here is this understanding correct? Are there any other problems I am not aware of?As a test of that understanding, #8255 says:
If
dns-lexiconor another project removed theirsetup.pyfile and we couldn't use their wheel, this approach of turning off pip's build isolation should continue to work as long as we have any dependencies, such aspoetry, installed in the build environment correct?If I'm understanding things correctly, my next question is if there is future work to be done here. Should we be disabling build isolation in other locations such as the DNS plugin snaps, Dockerfiles, CI config, or our scripts in
toolssuch as thevenv*.pyscripts or the release script? How will we ensure that build dependencies continue to be properly satisfied as we update the pinnings of our dependencies? Do we want to make a request or start a conversation with any of our dependencies about improvements here?Two ideas for the last question so far are:
pipfor a way to have it continue to install the PEP 518 style build dependencies for us with the ability to further constrain those dependencies with a constraints file. These dependencies could be installed either in an isolated environment or the normal installation environment, although I think the former is preferred because it'd allow us to pin tools likepoetrywithout unnecessarily adding it to our installation (or manually removing it after installing any packages that need it to build).