Skip to content

Fix up boadcasts, tests, and logging across the board#1400

Merged
NickCraver merged 27 commits intomasterfrom
craver/tests-and-broadcast
Mar 23, 2020
Merged

Fix up boadcasts, tests, and logging across the board#1400
NickCraver merged 27 commits intomasterfrom
craver/tests-and-broadcast

Conversation

@NickCraver
Copy link
Copy Markdown
Collaborator

@NickCraver NickCraver commented Mar 21, 2020

This changes does the following:

  • Moves the broadcast to be only before the master reconfiguration to both before and after - a fix following 88dcf0c to fix the gap.
  • Tweaks tests by overall lessening runtime (and thus build server time). Overall, fixes a few static timeouts to be conditional (so they short circuit faster if met), and brings some tests that were some variant of the above into RELEASE since they're safe now.
  • Changes UntilCondition to take a TimeSpan, just because clearer. Even though I IntelliSense completed .FromMinutes() earlier and watched it like an idiot for a while...I stand by this decision!
  • Locks the ITestOutputHelper writer because...that was jacked up in races:

off to the races


Note: a lot of the test changes are just optimizations to delays which allow longer but short-circuit sooner. The important changes are in broadcast and locking around the test runner. I can think of downsides to neither, but want some @mgravell eyes. This should resolve a lot of flaky-ness with local (and build agent) tests. Not all of it, but a lot of it!

Thie changes the broadcast to be only before the master reconfiguration to both before *and* after - a fix following 88dcf0c to fix the gap.

Also fixes tests by overall lessening runtime (and thus buld server time), fixes a few static timeouts to be conditional (so they short circuit faster if met), and brings some tests that were some variant of the above into RELEASE since they're safe now.

Also, we're locking the output because...that was jacked up in races.
@ejsmith
Copy link
Copy Markdown
Contributor

ejsmith commented Mar 22, 2020

Looking good!

Nick Craver added 2 commits March 21, 2020 22:08
@NickCraver
Copy link
Copy Markdown
Collaborator Author

@mgravell Should be good for review - this isn't perfect but does fix more of the flaky. I propose we merge this so I can get the other PRs using fewer builds to get green - we're eating build minutes on flaky nonsense at the moment. This also just generally reduces the arbitrary pausing (instead spin checking the condition we want with a timeout) to eat fewer build minutes as well. Our builds basically doubled in expense when Sentinel landed due to these pauses. I should have paid more attention then.

Copy link
Copy Markdown
Collaborator

@mgravell mgravell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just the ToArray

@mgravell
Copy link
Copy Markdown
Collaborator

mgravell commented Mar 23, 2020

oh, ignore my suggestion; didn't realize that was a test; in that case: don't care about a ToArray; approved!

@NickCraver NickCraver merged commit b90f991 into master Mar 23, 2020
@NickCraver NickCraver deleted the craver/tests-and-broadcast branch March 23, 2020 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants