Skip to content

Pull missing docker images automatically#191

Merged
eseliger merged 8 commits into
masterfrom
pull-docker-images
Jun 9, 2020
Merged

Pull missing docker images automatically#191
eseliger merged 8 commits into
masterfrom
pull-docker-images

Conversation

@eseliger

@eseliger eseliger commented May 5, 2020

Copy link
Copy Markdown
Member

Closes #183

@eseliger eseliger added the action-exec Everything related to the action exec functionality label May 5, 2020
@eseliger eseliger requested a review from mrnugget May 5, 2020 00:47
@eseliger eseliger force-pushed the pull-docker-images branch from d6d27d5 to b3d6858 Compare May 5, 2020 00:49
@eseliger eseliger force-pushed the pull-docker-images branch from b3d6858 to 17b5afa Compare May 5, 2020 00:51
Comment thread cmd/src/actions_exec.go Outdated
Comment thread cmd/src/actions_exec.go Outdated
Comment thread cmd/src/actions_exec.go Outdated
Comment thread cmd/src/actions_exec.go Outdated
out, err := exec.CommandContext(ctx, "docker", "image", "inspect", "--format", "{{.Id}}", "--", image).CombinedOutput()
if err != nil {
return "", fmt.Errorf("error inspecting docker image (try `docker pull %q` to fix this): %s", image, bytes.TrimSpace(out))
// An error could mean the image just didn't exist, so we need to pull it first.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we determine that? What if the error is "docker: command does not exist"? We then log "Pulling image..." which would fail.

Is there are a way to determine whether we should pull the image or return the error?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic

  • tries to get the digest
  • on fail, tries to pull
  • tries to get the digest again
  • on fail, it's not related to the pull and we return the error

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I know that, but the "on fail" could be caused by anything, right?

If "tries to get the digest" fails because docker is not installed, then we run "docker pull" and print more stuff to stderr.

In other words: can't we check the output returned by "docker image inspect" to see if it contains "No such image" (or something) and only then try to pull?

Comment thread cmd/src/actions_exec.go Outdated
}
}()
errScanner := bufio.NewScanner(stderr)
go func() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of doing this whole dance with StdoutPipe/StderrPipe and the scanners, I think we can simplify this by moving some of that into the logger:

stdout := logger.InfoPipe()
pullCmd.Stdout = stdout

The InfoPipe and WarnPipe methods could return a prefixed io.Writer that only logs if the verbose mode is turned on.

Take a look at this method to get some ideas:

func (a *actionLogger) RepoStdoutStderr(repoName string) (io.Writer, io.Writer, bool) {
a.mu.Lock()
defer a.mu.Unlock()
w, ok := a.logWriters[repoName]
if !a.verbose {
return w, w, ok
}
stderrPrefix := fmt.Sprintf("%s -> [STDERR]: ", yellow.Sprint(repoName))
stderr := textio.NewPrefixWriter(os.Stderr, stderrPrefix)
stdoutPrefix := fmt.Sprintf("%s -> [STDOUT]: ", yellow.Sprint(repoName))
stdout := textio.NewPrefixWriter(os.Stderr, stdoutPrefix)
return io.MultiWriter(stdout, w), io.MultiWriter(stderr, w), ok
}

@eseliger

eseliger commented May 5, 2020

Copy link
Copy Markdown
Member Author

I reworked the log streaming, PTAL :)

@mrnugget mrnugget left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, but left some comments with suggestions.

Also, did you test it in different cases?

  1. src action exec -f action-with-docker-images-that-cannot-be-pulled.json
  2. src action exec -f action-with-docker-images-that-can-be-pulled.json
  3. src -v action exec -f action-with-docker-images-that-cannot-be-pulled.json
  4. src -v action exec -f action-with-docker-images-that-can-be-pulled.json

Comment thread cmd/src/actions_exec.go Outdated
Comment thread cmd/src/actions_exec.go Outdated
Comment thread cmd/src/actions_exec_logger.go Outdated
Comment thread cmd/src/actions_exec_logger.go
Comment thread cmd/src/actions_exec.go Outdated
Comment thread cmd/src/actions_exec.go Outdated
Comment thread cmd/src/actions_exec.go Outdated
eseliger and others added 5 commits June 8, 2020 17:49
Co-authored-by: Thorsten Ball <mrnugget@gmail.com>
# Conflicts:
#	cmd/src/actions_exec.go
#	cmd/src/actions_exec_logger.go
@eseliger eseliger requested review from mrnugget and ryanslade June 9, 2020 14:23
@eseliger

eseliger commented Jun 9, 2020

Copy link
Copy Markdown
Member Author

I finished this one up and seems to work well for me, can I get another review after I merged master into this branch? :)

Comment thread cmd/src/actions_exec.go Outdated
Co-authored-by: Thorsten Ball <mrnugget@gmail.com>
@eseliger eseliger merged commit 3b9ea70 into master Jun 9, 2020
@eseliger eseliger deleted the pull-docker-images branch June 9, 2020 15:44
scjohns pushed a commit that referenced this pull request Apr 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

action-exec Everything related to the action exec functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Automatically pull missing docker images

2 participants