This repository was archived by the owner on Sep 30, 2024. It is now read-only.
fix/sg: fix mangled log output from sg start and sg run#63405
Merged
Conversation
sg start and sg run
bobheadxi
commented
Jun 20, 2024
| <-start | ||
| err := stream.Stream(func(line string) { | ||
| mux.Lock() | ||
| out.Writef(lineFormat, output.StyleBold, color, name, output.StyleReset, line) |
Member
Author
There was a problem hiding this comment.
*output.Output is not concurrency-safe
Contributor
|
Seems fixed it! |
unknwon
approved these changes
Jun 20, 2024
Contributor
|
Thanks! |
jhchabran
referenced
this pull request
Jul 4, 2024
Fixes DINF-82; This was very much a rabbithole. A few things: - The race that @bobheadxi mentioned here https://github.com/sourcegraph/sourcegraph/pull/63405#discussion_r1648180713 wasn't from `*output.Output` being unsafe, but `outputtest.Buffer` as it happened again (see [DINF-82](https://linear.app/sourcegraph/issue/DINF-82/devsgsg-test-failed-with-a-detected-race-condition)) - There something messed up with `cmds.start()`, which sometimes ends up printing the command output _after_ the exit message instead of before. - The crude `sort.Strings(want|have)` that was there already fixes that. - And without the sleep, it's possible to read the output from the `outputtest.Buffer` before the command outputs get written to it. - The `time.Sleep(300 * time.Milliseconds)` _mitigates/hides_ that problem. At least, this shouldn't blow up in CI and buys us time to fix the whole thing. We're tracking this in DINF-104. And out of 200 runs, I also stumbled on a race in `progress_tty`, tracked in DINF-105 (that packages is originally meant to be used by `src-cli` and was re-used for `sg` 3 years ago). I'm pretty unhappy about the solution, but a bandage is better than nothing. While ideally, we should really reconsider dropping `std.Output` entirely in `sg` and use the good stuff from github.com/charmbracelet instead because we don't want to spend too much time on arcane terminal things ourselves, I'm much more about concerned the concurrency issues mentioned above. ## Test plan CI + `sg bazel test //dev/sg:sg_test --runs_per_test=100`
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Right now
sg run/sg startcan horribly mangle multi-line output. A nicely annotated report from @unknwon:Replacing the "buffered process logger" thing with https://github.com/bobheadxi/streamline which powers
sourcegraph/runetc (fairly reliably if I do say so myself) fixes this for a few cases where I can reliably repro wonky misordered output 😁Test plan
sg start dotcomwithsg.config.overwrite.yaml:Log scope
pgx.devtraceris consistently formatted ✅ , even with high volume of logsAlso don't see anything suspicious happening after running for a while