updated dockerfile to add python 3.12 by auvipy · Pull Request #8614 · celery/celery

auvipy · 2023-11-07T08:00:47Z

Update tooling. something could be tweaked/ removed from this PR

auvipy · 2023-11-07T08:03:23Z

I think we need some adjustment here https://github.com/celery/celery/blob/main/.github/workflows/docker.yml#L12 to trigger docker CI?

auvipy · 2023-11-07T08:08:41Z

I am not sure why the docker CI is not triggered

Nusnus · 2023-11-07T08:10:01Z

I think we need some adjustment here https://github.com/celery/celery/blob/main/.github/workflows/docker.yml#L12 to trigger docker CI?

Yeah you're right, can you fix it in this PR so we can see the build passes?

auvipy · 2023-11-07T08:10:37Z

I can

auvipy · 2023-11-07T08:15:22Z

yes and first lets see if it can parse now otherwise will add dockerfile path too

auvipy · 2023-11-07T08:18:06Z

#11 [ 6/23] RUN curl https://pyenv.run/ | bash
#11 0.222 % Total % Received % Xferd Average Speed Time Time Time Current
#11 0.222 Dload Upload Total Spent Left Speed
#11 0.223
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
#11 0.769 curl: (77) error setting certificate file: /etc/ssl/certs/ca-certificates.crt
#11 DONE 0.8s

#12 [ 7/23] RUN pyenv install 3.8
#12 0.341 /bin/sh: 1: pyenv: not found
#12 ERROR: process "/bin/sh -c pyenv install 3.8" did not complete successfully: exit code: 127

[ 7/23] RUN pyenv install 3.8:
0.341 /bin/sh: 1: pyenv: not found

Dockerfile:66

64 |
65 | # Install required Python versions
66 | >>> RUN pyenv install 3.8
67 | RUN pyenv install 3.9
68 | RUN pyenv install 3.10

ERROR: failed to solve: process "/bin/sh -c pyenv install 3.8" did not complete successfully: exit code: 127
Service 'celery' failed to build : Build failed
make: *** [Makefile:179: docker-build] Error 1

auvipy · 2023-11-07T08:21:43Z

I got the source of error

Nusnus · 2023-11-07T08:21:51Z

docker/Dockerfile


 # Pypy3 is installed from a package manager because it takes so long to build.
-RUN apt-get update && apt-get install -y build-essential \
+RUN apt-get update && apt-get install --no-install-recommends -y build-essential \


Are you sure this change was needed?

bit sure but willl cross check and remove if these are the root cause

@auvipy yeah this line breaks the build.
I just tried locally, with/without to verify.

Ok so removing it passes a certain stage it didn't with it, but the build just failed locally.
I'll try to see what could have caused it

6206a35 I added the package for fixing certificate error

Nusnus · 2023-11-07T08:22:02Z

docker/Dockerfile

    libssl-dev \
-    sudo
+    sudo \
+    && apt-get clean


Also this is not needed and if so without the apt-get again

Nusnus

I suggest leaving the changes you made to add Python 3.12 and remove the rest

Nusnus · 2023-11-07T09:15:39Z

docker/scripts/install-pyenv.sh

Good catch!

auvipy · 2023-11-07T10:16:09Z

#26 634.8 no such option: -m
#26 ERROR: process "/bin/sh -c pyenv exec python3.12 -m pip install -r

Nusnus · 2023-11-07T11:11:18Z

Passed!! @auvipy 🎉

Nusnus · 2023-11-07T11:12:26Z

I'll test it locally too using make docker-build, 1 sec

auvipy · 2023-11-07T11:15:07Z

yes sure please share / tell me where the make file is? can't find it for some reason

auvipy · 2023-11-07T11:15:46Z

we can wait before merge. let me now how is it going

Nusnus · 2023-11-07T11:39:15Z

@auvipy works locally!

make docker-build

[+] Building 979.5s (41/41) FINISHED                                                                                                          docker:desktop-linux
 => [celery internal] load build definition from Dockerfile                                                                                                   0.0s
 => => transferring dockerfile: 4.92kB                                                                                                                        0.0s
 => [celery internal] load .dockerignore                                                                                                                      0.0s
 => => transferring context: 348B                                                                                                                             0.0s
 => [celery internal] load metadata for docker.io/library/debian:bookworm-slim                                                                                1.9s
 => [docs internal] load build definition from Dockerfile                                                                                                     0.0s
 => => transferring dockerfile: 935B                                                                                                                          0.0s
 => [docs internal] load .dockerignore                                                                                                                        0.0s
 => => transferring context: 348B                                                                                                                             0.0s
 => [docs internal] load metadata for docker.io/library/python:3.9-slim-bullseye                                                                              1.9s
 => [docs 1/8] FROM docker.io/library/python:3.9-slim-bullseye@sha256:d67a8b8773a6590cf440f597cb0d152e86242f7e866b57c98d60ee2936c4561f                        0.0s
 => [docs internal] load build context                                                                                                                        0.0s
 => => transferring context: 2.46kB                                                                                                                           0.0s
 => CACHED [celery  1/23] FROM docker.io/library/debian:bookworm-slim@sha256:6cc67f78e0e8295b4fbe055eba0356648f149daf15db9179aa51fcfa9cc131cd                 0.0s
 => [celery internal] load build context                                                                                                                      0.1s
 => => transferring context: 80.99kB                                                                                                                          0.0s
 => [celery  2/23] RUN apt-get update && apt-get install -y build-essential     libcurl4-openssl-dev     libffi-dev     tk-dev     xz-utils     ca-certific  32.9s
 => CACHED [docs 2/8] RUN apt-get update     && apt-get install -y build-essential     && apt-get install -y texlive     && apt-get install -y texlive-latex  0.0s
 => CACHED [docs 3/8] COPY /requirements /requirements                                                                                                        0.0s
 => CACHED [docs 4/8] RUN pip install -r /requirements/docs.txt -r /requirements/default.txt                                                                  0.0s
 => CACHED [docs 5/8] COPY docker/docs/start /start-docs                                                                                                      0.0s
 => CACHED [docs 6/8] RUN sed -i 's/\r$//g' /start-docs                                                                                                       0.0s
 => CACHED [docs 7/8] RUN chmod +x /start-docs                                                                                                                0.0s
 => CACHED [docs 8/8] WORKDIR /docs                                                                                                                           0.0s
 => [docs] exporting to image                                                                                                                                 0.0s
 => => exporting layers                                                                                                                                       0.0s
 => => writing image sha256:1def30e2bf695b46c1de365cd0bce09198583fcdee1ef7cad288c43b60585d37                                                                  0.0s
 => => naming to docker.io/celery/docs                                                                                                                        0.0s
 => [celery  3/23] RUN : "developer"                                                                                                                          0.2s
 => [celery  4/23] WORKDIR /provisioning                                                                                                                      0.1s
 => [celery  5/23] RUN useradd -m -s /bin/bash developer                                                                                                      0.1s
 => [celery  6/23] RUN curl https://pyenv.run | bash                                                                                                         12.3s
 => [celery  7/23] RUN pyenv install 3.12                                                                                                                    44.9s
 => [celery  8/23] RUN pyenv install 3.11                                                                                                                    46.0s
 => [celery  9/23] RUN pyenv install 3.10                                                                                                                    49.4s
 => [celery 10/23] RUN pyenv install 3.9                                                                                                                     45.1s
 => [celery 11/23] RUN pyenv install 3.8                                                                                                                     44.0s
 => [celery 12/23] RUN pyenv global 3.12 3.11 3.10 3.9 3.8                                                                                                    0.2s
 => [celery 13/23] WORKDIR /home/developer                                                                                                                    0.0s
 => [celery 14/23] COPY --chown=1000:1000 requirements /home/developer/requirements                                                                           0.0s
 => [celery 15/23] COPY --chown=1000:1000 docker/entrypoint /entrypoint                                                                                       0.0s
 => [celery 16/23] RUN chmod gu+x /entrypoint                                                                                                                 0.1s
 => [celery 17/23] RUN pyenv local 3.12 3.11 3.10 3.9 3.8                                                                                                     0.3s
 => [celery 18/23] RUN pyenv exec python3.12 -m pip install --upgrade pip setuptools wheel &&     pyenv exec python3.11 -m pip install --upgrade pip setupt  16.8s
 => [celery 19/23] COPY --chown=1000:1000 . /home/developer/celery                                                                                            0.2s
 => [celery 20/23] RUN pyenv exec python3.12 -m pip install -e /home/developer/celery &&     pyenv exec python3.11 -m pip install -e /home/developer/celery  39.1s
 => [celery 21/23] RUN pyenv exec python3.12 -m pip install -r requirements/default.txt   -r requirements/dev.txt   -r requirements/docs.txt   -r requirem  633.8s
 => [celery 22/23] WORKDIR /home/developer/celery                                                                                                             0.0s
 => [celery 23/23] RUN git config --global --add safe.directory /home/developer/celery                                                                        0.2s
 => [celery] exporting to image                                                                                                                              11.8s
 => => exporting layers                                                                                                                                      11.8s
 => => writing image sha256:ee1772b5f995ced4219d008c6d145c7180fad499588e7f500fb6414a59bb536f                                                                  0.0s
 => => naming to docker.io/celery/celery:dev                                                                                                                  0.0s

auvipy · 2023-11-07T11:41:57Z

oh yeah! thanks for your continuous support! I had a question here #8614 (comment)

Nusnus · 2023-11-07T11:57:43Z

@auvipy

celery/Makefile

Line 57 in 308255e

@echo "Docker-specific commands:"

auvipy · 2023-11-07T11:58:38Z

😁

* updated dockerfile to add python 3.12 * try to trigger docker CI by pointing to right directory * re organise versions and links * added ca-certificates to fix cert error * re order

updated dockerfile to add python 3.12

6a4bbba

auvipy requested a review from Nusnus November 7, 2023 08:01

auvipy added the Component: Docker Image label Nov 7, 2023

auvipy added this to the 5.3.x milestone Nov 7, 2023

Nusnus approved these changes Nov 7, 2023

View reviewed changes

try to trigger docker CI by pointing to right directory

2aaa09c

Nusnus reviewed Nov 7, 2023

View reviewed changes

re organise versions and links

77b53f7

Nusnus requested changes Nov 7, 2023

View reviewed changes

Nusnus reviewed Nov 7, 2023

View reviewed changes

docker/scripts/install-pyenv.sh

Copy link

Member

Nusnus Nov 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

added ca-certificates to fix cert error

6206a35

re order

4604f99

Nusnus approved these changes Nov 7, 2023

View reviewed changes

auvipy marked this pull request as ready for review November 7, 2023 11:15

Nusnus merged commit 2a4c427 into main Nov 7, 2023

auvipy deleted the dokpy312 branch November 7, 2023 11:42

Uh oh!

Conversation

auvipy commented Nov 7, 2023

Uh oh!

auvipy commented Nov 7, 2023

Uh oh!

auvipy commented Nov 7, 2023

Uh oh!

Nusnus commented Nov 7, 2023

Uh oh!

auvipy commented Nov 7, 2023

Uh oh!

auvipy commented Nov 7, 2023

Uh oh!

auvipy commented Nov 7, 2023

#12 [ 7/23] RUN pyenv install 3.8 #12 0.341 /bin/sh: 1: pyenv: not found #12 ERROR: process "/bin/sh -c pyenv install 3.8" did not complete successfully: exit code: 127

Dockerfile:66

64 | 65 | # Install required Python versions 66 | >>> RUN pyenv install 3.8 67 | RUN pyenv install 3.9 68 | RUN pyenv install 3.10

Uh oh!

auvipy commented Nov 7, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Nusnus left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

auvipy commented Nov 7, 2023

Uh oh!

Nusnus commented Nov 7, 2023

Uh oh!

Nusnus commented Nov 7, 2023

Uh oh!

auvipy commented Nov 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

auvipy commented Nov 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Nusnus commented Nov 7, 2023

Uh oh!

auvipy commented Nov 7, 2023

Uh oh!

Nusnus commented Nov 7, 2023

Uh oh!

auvipy commented Nov 7, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

#12 [ 7/23] RUN pyenv install 3.8
#12 0.341 /bin/sh: 1: pyenv: not found
#12 ERROR: process "/bin/sh -c pyenv install 3.8" did not complete successfully: exit code: 127

64 |
65 | # Install required Python versions
66 | >>> RUN pyenv install 3.8
67 | RUN pyenv install 3.9
68 | RUN pyenv install 3.10

auvipy commented Nov 7, 2023 •

edited

Loading

auvipy commented Nov 7, 2023 •

edited

Loading