Skip to content

Docker file changes, shell script changes#21

Merged
zocker-160 merged 31 commits intomasterfrom
fix-machine-uuid-overriding
Nov 6, 2023
Merged

Docker file changes, shell script changes#21
zocker-160 merged 31 commits intomasterfrom
fix-machine-uuid-overriding

Conversation

@Jeducious
Copy link
Copy Markdown
Contributor

Some Big Changes!

Motivation

The main motivation here is to make the docker system more flexible, this means allowing users to build images themselves using whatever base image and version of Blender they wish. It also means allowing for using development branches of the addon inside a running container via a bind mount. The really helps include docker images in our development cycle as currently the only way to test if new code in the addon works in a container is to publish the addon live, and then pull it into a running container.

Changes

Dockerfile

  • Make it possible to build a functioning Blender/Crowdrender image from a base image like debian/Ubuntu/Nvidia etc. Optinos are available to choose the versino of Blender to install into the image.
  • Removed the 'local' option, this is replaced by passing through the following options as env vars to the
    running container; persistent; token; secret; machine_uuid

Shell scripts

  • Support the new --override-machine-uuid option for crowdrender.
  • Allow the addon to be mounted from a bind mount so that a development
    branch of Crowdrender can be used inside a container. This is via a new shell script run_dev_env.sh
  • Modified the install_addon script so that its possible ot run crowdrender from a bind mount
    instead of downloading a version from discovery.

Other Chnages

  • Added a dockerignore file
  • Updated the make file to support docker building with other veresions of Blender.
  • Updated readme
  • Updated the github actions file to include options for building images with specified
    versions of blender.
    Separately the secrets for the docket name and token have been changed on github to point to the new crowdrender docker hub.

Dockerfile
- Make it possible to build a functioning Blender/Crowdrender image from a base image like debian/Ubuntu/Nvidia etc. Optinos are available to choose the versino of Blender to install into the image.
- Removed the 'local' option, this is replaced by passing through the following options as env vars to the
   running container; persistent; token; secret; machine_uuid

Shell scripts
 - Support the new --override-machine-uuid option for crowdrender.
 - Allow the addon to be mounted from a bind mount so that a development
    branch of Crowdrender can be used inside a container.
 - Modified the install_addon script so that its possible ot run crowdrender from a bind mount
   instead of downloading a version from discovery.

Other Chnages
 - Added a dockerignore file
 - Updated the make file to support docker building with other veresions of Blender.
 - Updated readme
 - Updated the github actions file to include options for building images with specified
    versions of blender.
Separately the secrets for the docket name and token have been changed on github to point to the new crowdrender docker hub.
@Jeducious Jeducious requested a review from zocker-160 August 9, 2022 03:10
Copy link
Copy Markdown
Collaborator

@zocker-160 zocker-160 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, there are a few blockers and a lot of nitpicks.

Also it would be good if you rebased this on top of master in order to have the updated readme, since you still have the old instructions for Nvidia users in this branch.

Jeducious and others added 18 commits August 30, 2022 20:57
The Blender download URL was specified as a separate input to the github action file. This is changed so that now the URL is constructed from other variables, namely the short and long version string, allowing you to customise the version of Blender used.

I also removed a build arg from the make file as it was overriding the version number with nothing causing a build error.
Dockerfile
- Make it possible to build a functioning Blender/Crowdrender image from a base image like debian/Ubuntu/Nvidia etc. Optinos are available to choose the versino of Blender to install into the image.
- Removed the 'local' option, this is replaced by passing through the following options as env vars to the
   running container; persistent; token; secret; machine_uuid

Shell scripts
 - Support the new --override-machine-uuid option for crowdrender.
 - Allow the addon to be mounted from a bind mount so that a development
    branch of Crowdrender can be used inside a container.
 - Modified the install_addon script so that its possible ot run crowdrender from a bind mount
   instead of downloading a version from discovery.

Other Chnages
 - Added a dockerignore file
 - Updated the make file to support docker building with other veresions of Blender.
 - Updated readme
 - Updated the github actions file to include options for building images with specified
    versions of blender.
Separately the secrets for the docket name and token have been changed on github to point to the new crowdrender docker hub.
The Blender download URL was specified as a separate input to the github action file. This is changed so that now the URL is constructed from other variables, namely the short and long version string, allowing you to customise the version of Blender used.

I also removed a build arg from the make file as it was overriding the version number with nothing causing a build error.
Removing refs to deprecated Dockerfiles, updating tags as well
Tested on windows, running WSL2, I can render using an RTX GPU in a container. Seems like the container toolkit isn't required.
Supporting the building of multiple Blender versions is now a lot easier.

I used the matrix builds in actions to create a list of blender versions that will be supported.

Some minor changes to the naming of the repository and the tags are in here too.
Only need the filepath to the addon package now, other options have been removed.

Support for using a copy of the addon that is on the host, but not in the container is done by simply omitting the -i flag, and making sure the local copy is bound to the container using docker's bind mounts.
The sript called for both an image name and a tag, its simpler to just accept the image name with the tag included in it.

Removed checking for this tag as well.

Added a variable to hold the path ot the location the addon should be installed when running a dev session. All dev sessions use a copy of the addon hosted on the host, not in the container, hence the use of a bind mount.
The new V0.6.4 and beyond will reequire a change to the commands used to start a server/worker.

Also, the var "use_local_cr" is now false by default and I've made some effort to consolidate and tidy up.
Docker interprets ENV key value such that the value is always a string, no need for explicit quotation marks.
Nvidia seems to have moved drivers into the toolkit, the cuda/driver images are depracated.

Needs testing on a system that has a GPU though.
Reflects changes to the tags, repo name, supported versions of Blender.

Two sections now, one for support of Docker Desktop and one for linux.
Better reflects the intended target system.
@Jeducious
Copy link
Copy Markdown
Contributor Author

Hi @zocker-160, I've just pushed commits to this branch/PR. Would really appreciate if you can let me know what you think and if we can merge. I really would like to get these containers back in use, since we've not updated them in ages, I think people are less likely to use them since Blender has moved on and is about to move to V4.0.

I think the changes I've made will help, I've tested GPU support on Docker Desktop in windows with a new docker file that is based on the original. All works well.

Cheers!

@Jeducious Jeducious requested a review from zocker-160 November 3, 2023 03:47
Should have supervised the AI better.
matrix vars not available to b used in the command to build the blender url resulted in a useless URL and no blender being downloaded.
this package isn't included in slim, so adding it.
Was using matrix builds incorrectly, resulting in the Blender URL being wrong.

I've now paired up the short and full version names so that the builds use the correct short and full versions when constructing the blender download URL.
Forgot to carry changes from previous commit to the same lines in the docker desktop build.
Wrong name for the image in the build command.
Copy link
Copy Markdown
Collaborator

@zocker-160 zocker-160 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small changes needed to make base image run on native Linux machine.

@zocker-160
Copy link
Copy Markdown
Collaborator

zocker-160 commented Nov 6, 2023

Seems to be working from my short testing, merging now after our conversation on Slack, but more changes will be needed (especially in the Readme which is kinda a mess IMO).

@zocker-160 zocker-160 merged commit 1a338b1 into master Nov 6, 2023
@Jeducious Jeducious deleted the fix-machine-uuid-overriding branch November 7, 2023 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants