You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thank you for submitting an issue. Please refer to our issue policy for additional information about bug reports. For help with debugging your code, please refer to Stack Overflow.
Please fill in this bug report template to ensure a timely and thorough response.
Willingness to contribute
The MLflow Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the MLflow code base?
Yes. I can contribute a fix for this bug independently.
Yes. I would be willing to contribute a fix for this bug with guidance from the MLflow community.
No. I cannot contribute a bug fix at this time.
System information
Have I written custom code (as opposed to using a stock example script provided in MLflow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
MLflow installed from (source or binary): pip
MLflow version (run mlflow --version): 1.18.0
Python version: 3.7.10
npm version, if running the dev UI:
Exact command to reproduce: mlflow run examples/docker -P alpha=0.5 or mlflow run examples/docker -P alpha=0.5 --backend kubernetes --backend-config kubernetes_config.json
Describe the problem
Describe the problem clearly here. Include descriptions of the expected behavior and the actual behavior.
I work on my company laptop which may be the reason this bug happened. I have been trying to use MLflow projects in Docker environment or with Kubernetes as backend. However when I try to run it, it produces PermissionError. The error is produced whenever MLflow tries to remove the temp directory it created in _create_docker_build_ctx function. this is the exact line that produces the error
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
Clone MLflow repository and try to run the docker container with the following commands:
mlflow run examples/docker -P alpha=0.5
mlflow run examples/docker -P alpha=0.5 --backend kubernetes --backend-config kubernetes_config.json
Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
Traceback (most recent call last):
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\xx\Anaconda3\envs\mlflow-projects\Scripts\mlflow.exe_main.py", line 7, in
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\site-packages\click\core.py", line 1137, in call
return self.main(*args, **kwargs)
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\site-packages\click\core.py", line 1062, in main
rv = self.invoke(ctx)
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\site-packages\click\core.py", line 1668, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\site-packages\click\core.py", line 763, in invoke
return callback(*args, **kwargs)
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\site-packages\mlflow\cli.py", line 181, in run
run_id=run_id,
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\site-packages\mlflow\projects_init.py", line 304, in run
synchronous=synchronous,
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\site-packages\mlflow\projects_init.py", line 99, in _run
experiment_id,
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\site-packages\mlflow\projects\backend\local.py", line 76, in run
run_id=active_run.info.run_id,
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\site-packages\mlflow\projects\docker.py", line 65, in build_docker_image
build_ctx_path = _create_docker_build_ctx(work_dir, dockerfile)
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\site-packages\mlflow\projects\docker.py", line 137, in _create_docker_build_ctx
shutil.rmtree(directory)
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\shutil.py", line 516, in rmtree
return _rmtree_unsafe(path, onerror)
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\shutil.py", line 395, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\shutil.py", line 395, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\shutil.py", line 395, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
[Previous line repeated 1 more time]
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\shutil.py", line 400, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "c:\users\xx\anaconda3\envs\mlflow-projects\lib\shutil.py", line 398, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\Users\xxx\AppData\Local\Temp\tmpzm3h9gtc\mlflow-project-contents\.git\objects\01\901b5494e36bbb8c92437dd4d69853a6d70469'
What component(s), interfaces, languages, and integrations does this bug affect?
Components
area/artifacts: Artifact stores and artifact logging
area/build: Build and test infrastructure for MLflow
area/docs: MLflow documentation pages
area/examples: Example code
area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
area/models: MLmodel format, model serialization/deserialization, flavors
area/projects: MLproject format, project running backends
area/scoring: Local serving, model deployment tools, spark UDFs
area/server-infra: MLflow server, JavaScript dev server
Thank you for submitting an issue. Please refer to our issue policy for additional information about bug reports. For help with debugging your code, please refer to Stack Overflow.
Please fill in this bug report template to ensure a timely and thorough response.
Willingness to contribute
The MLflow Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the MLflow code base?
System information
mlflow --version): 1.18.0mlflow run examples/docker -P alpha=0.5ormlflow run examples/docker -P alpha=0.5 --backend kubernetes --backend-config kubernetes_config.jsonDescribe the problem
Describe the problem clearly here. Include descriptions of the expected behavior and the actual behavior.
I work on my company laptop which may be the reason this bug happened. I have been trying to use MLflow projects in Docker environment or with Kubernetes as backend. However when I try to run it, it produces
PermissionError. The error is produced whenever MLflow tries to remove the temp directory it created in_create_docker_build_ctxfunction. this is the exact line that produces the errormlflow/mlflow/projects/docker.py
Line 117 in 221fedc
Code to reproduce issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
Clone MLflow repository and try to run the docker container with the following commands:
mlflow run examples/docker -P alpha=0.5mlflow run examples/docker -P alpha=0.5 --backend kubernetes --backend-config kubernetes_config.jsonOther info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
What component(s), interfaces, languages, and integrations does this bug affect?
Components
area/artifacts: Artifact stores and artifact loggingarea/build: Build and test infrastructure for MLflowarea/docs: MLflow documentation pagesarea/examples: Example codearea/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registryarea/models: MLmodel format, model serialization/deserialization, flavorsarea/projects: MLproject format, project running backendsarea/scoring: Local serving, model deployment tools, spark UDFsarea/server-infra: MLflow server, JavaScript dev serverarea/tracking: Tracking Service, tracking client APIs, autologgingInterface
area/uiux: Front-end, user experience, JavaScript, plottingarea/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Modelsarea/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registryarea/windows: Windows supportLanguage
language/r: R APIs and clientslanguage/java: Java APIs and clientslanguage/new: Proposals for new client languagesIntegrations
integrations/azure: Azure and Azure ML integrationsintegrations/sagemaker: SageMaker integrationsintegrations/databricks: Databricks integrations