Skip to content

fix: Race condition in log streaming#242

Merged
yevgenypats merged 8 commits intomainfrom
fix-log-streaming-race
Oct 4, 2022
Merged

fix: Race condition in log streaming#242
yevgenypats merged 8 commits intomainfrom
fix-log-streaming-race

Conversation

@hermanschaaf
Copy link
Copy Markdown
Contributor

This adds a sync.WaitGroup to ensure that we wait for all logs to finish streaming before returning from Terminate. Otherwise, the log file might get closed before we get to write the final lines.

Copy link
Copy Markdown
Contributor

@yevgenypats yevgenypats left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really good catch. found one bug

Copy link
Copy Markdown
Member

@disq disq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need the same logic in the destinationclient as well.

@hermanschaaf hermanschaaf requested a review from disq October 4, 2022 15:23
Copy link
Copy Markdown
Member

@disq disq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still need the same fix in the destination client.

Copy link
Copy Markdown
Contributor

@yevgenypats yevgenypats left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it needs a few more tweaks. still race conditions here.

Copy link
Copy Markdown
Contributor

@yevgenypats yevgenypats left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commented on the kill side (prob I should just use the comments - sorry for the re-review)

}
if c.cmd != nil && c.cmd.Process != nil {
if err := c.cmd.Process.Kill(); err != nil {
// if we fail to kill the process, we also won't wait for logs to finish streaming
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this comment is correct or at least we need to see what are the error message from kill.

This is the snippet from signal, it really can only fail if it already closed. you can always kill a child process. so we do want to wait for the logs if it is already killed.

func (p *Process) signal(sig Signal) error {
	if p.Pid == -1 {
		return errors.New("os: process already released")
	}
	if p.Pid == 0 {
		return errors.New("os: process not initialized")
	}
	p.sigMu.RLock()
	defer p.sigMu.RUnlock()
	if p.done() {
		return ErrProcessDone
	}
	s, ok := sig.(syscall.Signal)
	if !ok {
		return errors.New("os: unsupported signal type")
	}
	if e := syscall.Kill(p.Pid, s); e != nil {
		if e == syscall.ESRCH {
			return ErrProcessDone
		}
		return e
	}
	return nil
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yevgenypats Updated

@yevgenypats yevgenypats merged commit 3c8242a into main Oct 4, 2022
@yevgenypats yevgenypats deleted the fix-log-streaming-race branch October 4, 2022 16:04
yevgenypats pushed a commit that referenced this pull request Oct 4, 2022
🤖 I have created a release *beep* *boop*
---


##
[0.12.4](v0.12.3...v0.12.4)
(2022-10-04)


### Bug Fixes

* Improve download message
([#240](#240))
([7929bbb](7929bbb))
* Race condition in log streaming
([#242](#242))
([3c8242a](3c8242a))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants