Skip to content

Update conda channel order to sync with Bioconda#4409

Merged
jmchilton merged 3 commits intogalaxyproject:devfrom
nsoranzo:fix_conda_channel_order
Aug 15, 2017
Merged

Update conda channel order to sync with Bioconda#4409
jmchilton merged 3 commits intogalaxyproject:devfrom
nsoranzo:fix_conda_channel_order

Conversation

@nsoranzo
Copy link
Member

@nsoranzo nsoranzo commented Aug 10, 2017

Also:

  • Remove _ensure_conda_context() function and make conda_context argument mandatory in functions/methods.
    This should help making sure the correct context (e.g. channels) is used for all Conda operations.
  • Remove unused is_target_available() function.
  • Override channel list on the command line instead of modifying condarc.
    The way we were modifying condarc in the ensure_channels_configured() method was broken because we were appending the missing channels at the end, so the final order may not be the one specified in conda_ensure_channels .
    Also not modifying the user condarc is a much better approach and overriding the channel list on the command line will make debugging easier.

Copy link
Member

@bgruening bgruening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nsoranzo thanks for doing this! It is somewhere on my todo list.

I think we also need to update planemo and galaxy-lib.
Do we want to leave this open until we have PRs for above mentioned PRs as well?

@nsoranzo
Copy link
Member Author

@bgruening Any idea why this integration test fails? https://jenkins.galaxyproject.org/job/docker-integration/1349/testReport/junit/integration.test_resolvers/CondaResolutionIntegrationTestCase/test_legacy_r_mapping/?auto_refresh=false

It seems it's mixing r-base from conda-forge with the broken readline-6.2-2 from the defaults channel instead of readline-6.2-0 from conda-forge.

@nsoranzo
Copy link
Member Author

xref. galaxyproject/planemo#715
I think that galaxy-lib will be updated when the changes from this galaxy repository are merged in it.

@bgruening
Copy link
Member

It looks like most of the packages are coming from default - this is wired. Most of them, if not all, should come from conda-forge.

@jmchilton
Copy link
Member

I'll take care of planemo and galaxy-lib after we merge this - but that test failure is scary. I think I pulled that example from an actual tool that needed the merge Conda installs.

@nsoranzo
Copy link
Member Author

@jmchilton Should we add a warning at Galaxy startup if the configured channel order is not the one in conda_ensure_channels ?

@nsoranzo
Copy link
Member Author

I think the issue is that the galaxy.tools.deps.conda_util.CondaContext.ensure_channels_configured() method appends the configured channels at the end, which by default is the defaults channel. So defaults remains the highest priority channel instead of being the lowest.

@nsoranzo
Copy link
Member Author

nsoranzo commented Aug 13, 2017

My plan to fix the problem here is to remove ensure_channels_configured(), load_condarc() and save_condarc() and instead override the channel list on the command line as in

search_cmd.append("--override-channels")
for channel in channels_override:
search_cmd.extend(["--channel", channel])

Please let me know if you have reservations.

@mvdbeek
Copy link
Member

mvdbeek commented Aug 13, 2017

I think that is the best approach @nsoranzo, no need to manipulate and conflict with users' .condarc

and make conda_context argument mandatory in functions/methods.
This should help making sure the correct context (e.g. channels)
is used for all Conda operations.

Also remove unused is_target_available() function.
The way we were modifying condarc in the ensure_channels_configured()
method was broken because we were appending the missing channels at the
end, so the final order may not be the one specified in
conda_ensure_channels .

Also:
- not modifying the user condarc is a much better approach
- overriding the channel list on the command line will make debugging
  easier
@nsoranzo nsoranzo force-pushed the fix_conda_channel_order branch from c802cbe to cdbac05 Compare August 14, 2017 14:50
@jmchilton jmchilton merged commit 44548f8 into galaxyproject:dev Aug 15, 2017
@nsoranzo nsoranzo deleted the fix_conda_channel_order branch August 15, 2017 14:15
jmchilton added a commit that referenced this pull request Oct 2, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants