-
Notifications
You must be signed in to change notification settings - Fork 634
Description
Snakemake version
8.20.3
Describe the bug
When using a storage plugin e.g. snakemake-storage-plugin-xrootd the user can provide a username/password or encode an authentication token in the file path. When snakemake then prints the DAG or error messages the full path, including the token/username/password whatever, is printed in the terminal exposing potentially sensitive information.
It would be great to have a way to be able to ensure that only ever a censored form of the URL is printed (but still allow for the full, uncensored URL to be passed as input/output to the shell part).
Minimal example
storage:
provider="xrootd",
host="eoslhcb.cern.ch",
username="my_username"
rule test_rule:
input: storage("root://eos/my_file.root")
output: "test.flag"
shell: "touch {output}"It will correctly fail due to the file not existing and incorrect credentials:
$ snakemake -c1 test.flag -n --storage-xrootd-password="my_password"
A password has been specified -- it will be printed in plaintext when Snakemake displays the inputs/outputs of jobs! Only use this option in trusted environments.
Building DAG of jobs...
Error checking existence of root://my_username:****@eoslhcb.cern.ch:1094//eos/my_file.root on XRootD: [ERROR] Server responded with an error: [3010] Unable to give access - user access restricted - unauthorized identity used ; Permission denied
, attempt 1/3 failed - retrying in 3 seconds...
Unrecoverable error, no more retries
WorkflowError:
Failed to check existence of root://my_username:my_password@eoslhcb.cern.ch:1094//eos/my_file.root
XRootDFatalException: Error checking existence of root://my_username:****@eoslhcb.cern.ch:1094//eos/my_file.root on XRootD: [ERROR] Server responded with an error: [3010] Unable to give access - user access restricted - unauthorized identity used ; Permission denied
where you can see (more than once) the password would be printed in plain text.
Additional context
The snakemake-storage-plugin-xrootd has this postprocess_query method which can be changed to print out a censored URL instead but, as far as I can tell, this then prevents the plugin from working correctly because the censored URL is used as the "file" rather than actual full URL.
Our particular use-case for this is running CI/CD tests where the authentication is handeled by appending a token into the URL parameters and we would prefer not to be exposing that in the CI logs if we can avoid it!
Metadata
Metadata
Assignees
Labels
Type
Projects
Status