@mvdbeek Should I make a test with a workflow with an invalid tool? If yes, do you have an invalid tool to propose? Or should I use a tool which does not exists?

Regarding the other aspect: test if a repo is in the toolshed. Should it be part of planemo lint? or planemo test?
Any idea where I should insert the corresponding code block? Also we need to be sure that people that want to test a workflow with custom tools will be able to do it.

Or should I use a tool which does not exists?

Yep, I think that's best

Also we need to be sure that people that want to test a workflow with custom tools will be able to do it.

👍 We've hardcoded the main tool shed in the update code and we skip tools that don't come from a tool shed.

Should it be part of planemo lint?

I think planemo workflow_lint is more appropriate. We have a separate entry point for workflow linting in https://github.com/galaxyproject/planemo/blob/master/planemo/commands/cmd_workflow_lint.py. To implement the check you could add lint_context.lint("lint_tool_ids", _lint_tool_ids, potential_workflow_artifact_path) after

That function could skip local tools (that don't have /repos/ in the tool id).

Oh, and _lint_tool_ids could share code with def outdated_tools from planemo/autoupdate.py, you could move

        if not step["tool_id"].startswith(AUTOUPDATE_TOOLSHED_URL[8:]):
            return {}  # assume a built in tool
        try:
            repos = ts.repositories._get(params={"tool_ids": step["tool_id"]})
        except Exception:
            ctx.log(f"The ToolShed returned an error when searching for the most recent version of {step['tool_id']}")
            return {}

into a function ... in workflow_lint.py that autoupdate can import ?

So for info, if the tool is not in the toolshed:

$ planemo autoupdate workflow_with_unexisting_tool.ga 
bioblend ERROR: GET: error 500: b'{"err_msg": "Uncaught exception in exposed API method:", "err_code": 0}', 0 attempts left
The ToolShed returned an error when searching for the most recent version of toolshed.g2.bx.psu.edu/repos/lldelisle/unexisting/unexisting/1.1.0
No newer tool versions were found, so the workflow was not updated.

But the issue is when there is a version in the toolshed which is not the one that is in the workflow.

But the issue is when there is a version in the toolshed which is not the one that is in the workflow.

Ah, right!

I am close to it.

Should it be a fail or a warn?

I think fail is appropriate if the tool id looks like a tool shed id but is not found in the tool shed

I think it is safer if I write another PR.

Thanks

Thank you!