Skip to content

[RELAND] .circleci: Improve docker image build workflow#38279

Closed
seemethere wants to merge 1 commit intopytorch:masterfrom
seemethere:reland_rework_docker_ci_build_flow
Closed

[RELAND] .circleci: Improve docker image build workflow#38279
seemethere wants to merge 1 commit intopytorch:masterfrom
seemethere:reland_rework_docker_ci_build_flow

Conversation

@seemethere
Copy link
Copy Markdown
Member

@seemethere seemethere commented May 11, 2020

This reverts commit 7c2853b.

@seemethere seemethere requested a review from malfet May 11, 2020 23:09
@seemethere seemethere changed the title [RELAND] .circleci: Improve docker image build workflow WIP [RELAND] .circleci: Improve docker image build workflow May 11, 2020
@seemethere seemethere marked this pull request as draft May 11, 2020 23:18
@dr-ci
Copy link
Copy Markdown

dr-ci Bot commented May 11, 2020 •
edited
Loading

💊 CI failures summary and remediations

As of commit 49daeb8 (more details on the Dr. CI page):

1 failure not recognized by patterns:

Job Step Action
CircleCI docker-pytorch-linux-xenial-py3-clang5-android-ndk-r19c build_docker_image_pytorch-linux-xenial-py3-clang5-android-ndk-r19c 🔁 rerun

🚧 3 fixed upstream failures:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

Since your merge base is older than viable/strict, run these commands:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

Check out the recency history of this "viable master" tracking branch.

ci.pytorch.org: 1 failed

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

See how this bot performed.

This comment has been revised 6 times.

@seemethere seemethere marked this pull request as ready for review May 12, 2020 01:57
@seemethere seemethere changed the title WIP [RELAND] .circleci: Improve docker image build workflow [RELAND] .circleci: Improve docker image build workflow May 12, 2020
facebook-github-bot
facebook-github-bot reviewed May 12, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seemethere is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

malfet
malfet reviewed May 12, 2020
View reviewed changes
Comment thread .circleci/verbatim-sources/docker_jobs.yml
name: dynamically generate tags to keep
no_output_timeout: "1h"
command: |
GENERATED_IMAGE_TAG=$(\
Copy link
Copy Markdown
Contributor

@malfet malfet May 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As @ezyang suggested in original PR, can you add some explanation on what this command is doing?
I.e. generated image tag is SHA sum of last commit that touched .circleci/docker folder.

@malfet
Copy link
Copy Markdown
Contributor

malfet commented May 12, 2020

Is there any difference between this PR and #37976

This was referenced May 14, 2020
Closed
Closed
facebook-github-bot pushed a commit that referenced this pull request May 14, 2020
@seemethere @facebook-github-bot
.cirlceci: Keep tags that look like a sha1 (#38483)
3300dd5 
Summary:
Previous attempts to get this right:
* #38335
* #38279
* #37976

This tag kept getting deleted before the docker image ci workflow could
be merged causing it to have upstream breakages.

It'd be best to make sure the garbage collector just doesnt garbage
collect it.

This is a pre-step to merge #38484

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Pull Request resolved: #38483

Differential Revision: D21577359

Pulled By: seemethere

fbshipit-source-id: c4e0709bd8fff8f24a988b60eaa9f8c01576ef2f
facebook-github-bot pushed a commit that referenced this pull request May 15, 2020
@seemethere @facebook-github-bot
[RELAND] .circleci: Improve docker image build workflow (#38484)
8cdc480 
Summary:
closes #37855

Relies on #38483

Previous attempts to get this right:
* #38335
* #38279
* #37976

This reverts commit 8063960.

Improves the docker image build workflow from many steps to basically
transparent from a user's perspective.

To update docker images now all one has to do is edit the
.circleci/docker folder and it will update automatically and also
dynamically add the tags to the list of tags to keep from the garbage
collector.

Adding a new image will currently stay the same but we can explore doing
that dynamically as well.

How the build workflow works:
  - Docker tags are determined by the hash defined from git for the
    .circleci/docker sub-directory (extracted using git rev-parse)
  - Images are only built if the computed hash is not found in ecr and
    the hash is different than the previously computed hash. The
    previously computed hash is found using the same process as before
    but subbing out HEAD for the merge base between HEAD and the base
    git revision
  - That tag is then passed through the jobs using a shared workspace
    which is added to downstream jobs using the circleci ${BASH_ENV}

How the new garbage collection works:
  - Tags to keep are generated by stepping through all of the commits in
    in the .circleci/docker subdirectory

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Pull Request resolved: #38484

Differential Revision: D21585458

Pulled By: seemethere

fbshipit-source-id: 37792a1e0f5e5531438c4ae61507639c133aa76d
This was referenced May 18, 2020
Closed
Closed
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 24, 2026
@seemethere @facebook-github-bot
[RELAND] .circleci: Improve docker image build workflow (pytorch#38279)
c0cd59e 
Summary:
This reverts commit 6f02fc3.
Pull Request resolved: pytorch#38279

Differential Revision: D21517822

Pulled By: seemethere

fbshipit-source-id: 5f705f6c617cbc77a10ab9deb913bc5958ae7439
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 24, 2026
@seemethere @facebook-github-bot
.cirlceci: Keep tags that look like a sha1 (pytorch#38483)
98c778b 
Summary:
Previous attempts to get this right:
* pytorch#38335
* pytorch#38279
* pytorch#37976

This tag kept getting deleted before the docker image ci workflow could
be merged causing it to have upstream breakages.

It'd be best to make sure the garbage collector just doesnt garbage
collect it.

This is a pre-step to merge pytorch#38484

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Pull Request resolved: pytorch#38483

Differential Revision: D21577359

Pulled By: seemethere

fbshipit-source-id: c4e0709bd8fff8f24a988b60eaa9f8c01576ef2f
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 24, 2026
@seemethere @facebook-github-bot
[RELAND] .circleci: Improve docker image build workflow (pytorch#38484)
6c03011 
Summary:
closes pytorch#37855

Relies on pytorch#38483

Previous attempts to get this right:
* pytorch#38335
* pytorch#38279
* pytorch#37976

This reverts commit 46bc26c.

Improves the docker image build workflow from many steps to basically
transparent from a user's perspective.

To update docker images now all one has to do is edit the
.circleci/docker folder and it will update automatically and also
dynamically add the tags to the list of tags to keep from the garbage
collector.

Adding a new image will currently stay the same but we can explore doing
that dynamically as well.

How the build workflow works:
  - Docker tags are determined by the hash defined from git for the
    .circleci/docker sub-directory (extracted using git rev-parse)
  - Images are only built if the computed hash is not found in ecr and
    the hash is different than the previously computed hash. The
    previously computed hash is found using the same process as before
    but subbing out HEAD for the merge base between HEAD and the base
    git revision
  - That tag is then passed through the jobs using a shared workspace
    which is added to downstream jobs using the circleci ${BASH_ENV}

How the new garbage collection works:
  - Tags to keep are generated by stepping through all of the commits in
    in the .circleci/docker subdirectory

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Pull Request resolved: pytorch#38484

Differential Revision: D21585458

Pulled By: seemethere

fbshipit-source-id: 37792a1e0f5e5531438c4ae61507639c133aa76d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@malfet malfet malfet left review comments

@ezyang ezyang ezyang approved these changes

+1 more reviewer

@facebook-github-bot facebook-github-bot facebook-github-bot left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@seemethere @malfet @ezyang @facebook-github-bot