I'm submitting a...
Issues #793, #1696, and #3299 from different users all request that conda support optional dependencies. One interpretation of this request was implemented in conda 4.4, namely constraints for optional dependencies: packages listed under run_constrained are constrained to the specified version, but only if they are installed explicitly by the user.
The other interpretation of optional dependencies, i.e. like pip's groups of optional requirements that are specified using brackets like matplotlib[qt], remains unsupported. @pelson, @astrofrog, and others have provided arguments why it should be supported, but the response so far has been that conda metapackages cover this type of functionality. I'd like to challenge that claim.
It's true that a conda metapackage is in some sense an equivalent way to handle the functionality of pip's bracketed options, but in practice there are major different implications for the different approaches taken by pip and conda.
In pip, it's simple for a package maintainer to provide a long list of sets of options, such as the various optional backends available for matplotlib. No matter how many such options are provided, the result is only a single pip package to prepare and track for a given release, and only a single package that shows up in a listing of the environment. Thus in pip, listing such a set of relevant optional packages is very cheap for the developer, and also for the user, as it doesn't result in anything in the user's environment for them to track.
In conda, having to create a metapackage and track it in repositories, environments, licensing reviews, and so on very strongly discourages making such groups of optional dependencies. Sure, it can be done, but it's no longer trivial; it has substantial maintenance costs for the project maintainer and management costs for the user that the pip approach does not.
Perhaps more importantly for me as a package maintainer, this difference between conda and pip makes it much more difficult to maintain packages for both pip and conda, and creates ambiguities about what a package name means in any installation that combines both pip and conda. Specifically, pip's approach encourages the main package for a given library to be lean, without optional extras, while conda making it difficult to specify optional groups of things to install encourages packages to be fat, with any optional packages included as dependencies, even if they aren't strictly required for a minimal installation.
So e.g. we end up with some package "pack" on pip, with "pack[recommended]" installing the most typical dependencies, "pack[all]" installing lots of extra things, and "pack" installing only the bare minimum. On conda we maybe get "pack" installing the most typical dependencies, sometimes "pack-core" (a dependency of "pack") installing the bare minimum, and "pack-extras" installing the extra stuff.
So far the only problem is that we had to make and track three packages (and all their arch variants) where one would do. But more seriously, if a user installs "otherpack", which depends on "pack", she's now going to get a different set of dependencies if "otherpack" is installed with pip than if it's installed with conda (due to "pack" having different dependencies in the two cases). This leads to all sorts of trouble!
If conda would support a similar notation for installable groups of optional packages, project maintainers could safely support similar options on pip and conda without these constraints always being in tension, and users would see more predictable behavior.
I'm submitting a...
Issues #793, #1696, and #3299 from different users all request that conda support optional dependencies. One interpretation of this request was implemented in conda 4.4, namely constraints for optional dependencies: packages listed under
run_constrainedare constrained to the specified version, but only if they are installed explicitly by the user.The other interpretation of optional dependencies, i.e. like pip's groups of optional requirements that are specified using brackets like
matplotlib[qt], remains unsupported. @pelson, @astrofrog, and others have provided arguments why it should be supported, but the response so far has been that conda metapackages cover this type of functionality. I'd like to challenge that claim.It's true that a conda metapackage is in some sense an equivalent way to handle the functionality of pip's bracketed options, but in practice there are major different implications for the different approaches taken by pip and conda.
In pip, it's simple for a package maintainer to provide a long list of sets of options, such as the various optional backends available for matplotlib. No matter how many such options are provided, the result is only a single pip package to prepare and track for a given release, and only a single package that shows up in a listing of the environment. Thus in pip, listing such a set of relevant optional packages is very cheap for the developer, and also for the user, as it doesn't result in anything in the user's environment for them to track.
In conda, having to create a metapackage and track it in repositories, environments, licensing reviews, and so on very strongly discourages making such groups of optional dependencies. Sure, it can be done, but it's no longer trivial; it has substantial maintenance costs for the project maintainer and management costs for the user that the pip approach does not.
Perhaps more importantly for me as a package maintainer, this difference between conda and pip makes it much more difficult to maintain packages for both pip and conda, and creates ambiguities about what a package name means in any installation that combines both pip and conda. Specifically, pip's approach encourages the main package for a given library to be lean, without optional extras, while conda making it difficult to specify optional groups of things to install encourages packages to be fat, with any optional packages included as dependencies, even if they aren't strictly required for a minimal installation.
So e.g. we end up with some package "pack" on pip, with "pack[recommended]" installing the most typical dependencies, "pack[all]" installing lots of extra things, and "pack" installing only the bare minimum. On conda we maybe get "pack" installing the most typical dependencies, sometimes "pack-core" (a dependency of "pack") installing the bare minimum, and "pack-extras" installing the extra stuff.
So far the only problem is that we had to make and track three packages (and all their arch variants) where one would do. But more seriously, if a user installs "otherpack", which depends on "pack", she's now going to get a different set of dependencies if "otherpack" is installed with pip than if it's installed with conda (due to "pack" having different dependencies in the two cases). This leads to all sorts of trouble!
If conda would support a similar notation for installable groups of optional packages, project maintainers could safely support similar options on pip and conda without these constraints always being in tension, and users would see more predictable behavior.