You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When ty checks files in parallel, non-panic diagnostics with the same rendered source location, severity, and diagnostic ID can be printed in worker-completion order instead of a canonical order. This manifests as nondeterminism on the homeassistant/core project.
Root Cause
CollectReporter receives checked-file diagnostics as Rayon workers finish, then sorts them with Diagnostic::rendering_sort_key. The existing key compared primary file/range, severity, and diagnostic ID, but not the visible diagnostic message. In core, the two diagnostics had the same primary span and tied on error DiagnosticId::InvalidArgumentType, so the sort key considered visibly different diagnostics equal and Rust's stable sort preserved the race-dependent arrival order.
Fix
Use the concise rendered message as the final tie-breaker.
I had Codex look for nondeterminism. Sounds like it was checking projects that had previously seen nondeterminism and ran ty on homeassistant a bunch of times, found this nondeterminism, and root caused it.
I'd be curious to know why ecosystem-analyzer didn't detect the nondeterminism. It might be because ecosystem-analyzer sorts diagnostics internally before it does flakiness detection? Could be interesting to look into it as a followup
Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 91.94%. The percentage of expected errors that received a diagnostic held steady at 87.09%. The number of fully passing files held steady at 92/134.
I'd be curious to know why ecosystem-analyzer didn't detect the nondeterminism. It might be because ecosystem-analyzer sorts diagnostics internally before it does flakiness detection? Could be interesting to look into it as a followup
Sorry for the Codex noise on this, I asked it to investigate and it went a bit further than I wanted.
I think the answer is that ecosystem-analyzer sorts diagnostics internally, and therefore it hid the nondeterminism that this PR fixes. That's probably for the better; we should fix nondeterminism in diagnostic order but it's not a huge deal and we don't ecosystem-analyzer to do a side job in finding nondeterminism in ordering.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bugSomething isn't workingtyMulti-file analysis & type inference
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Nondeterminism
When ty checks files in parallel, non-panic diagnostics with the same rendered source location, severity, and diagnostic ID can be printed in worker-completion order instead of a canonical order. This manifests as nondeterminism on the homeassistant/core project.
Root Cause
CollectReporterreceives checked-file diagnostics as Rayon workers finish, then sorts them withDiagnostic::rendering_sort_key. The existing key compared primary file/range, severity, and diagnostic ID, but not the visible diagnostic message. Incore, the two diagnostics had the same primary span and tied on errorDiagnosticId::InvalidArgumentType, so the sort key considered visibly different diagnostics equal and Rust's stable sort preserved the race-dependent arrival order.Fix
Use the concise rendered message as the final tie-breaker.