client: Refresh cloud models on websocket reconnect#57528
Merged
Conversation
30726c1 to
56173cd
Compare
Until now, the cloud-hosted model list was only refreshed in response to events that exercise the LLM token (a `UserUpdated` push, an organization change, or `PrivateUserInfoUpdated`). If a user wasn't actively using AI features around the time we shipped new models, the list could stay stale until they restarted Zed. This is the second step toward fixing that, after #57078 made the cloud websocket reconnect on its own. We now treat each successful (re)connect as a hint that the server state may have changed, so possibly new model definitions will be available, and trigger a model list refresh. The trigger is a new `Client::cloud_connection_id()` watch that bumps a counter each time the websocket handshake completes. `CloudLanguageModelProvider::State` subscribes to it and, on every tick after the initial `0`, schedules a debounced refresh. The debounce is trailing-edge with a 5-minute window plus up to 5 minutes of uniform jitter, so a burst of reconnects (rolling deploy, flaky network) coalesces into a single refresh once things have been quiet, and we avoid thundering herd issues from many clients reconnecting at the same time. Closes CLO-713. Release Notes: - The list of Zed managed models is now refreshed automatically, without requiring a restart
56173cd to
fb3f4ac
Compare
tomhoule
commented
May 26, 2026
| /// | ||
| /// Each reconnect resets the timer (trailing-edge debounce), so a burst of reconnects | ||
| /// only triggers a single refresh after things have been quiet for this duration. | ||
| const MODELS_REFRESH_DEBOUNCE: Duration = Duration::from_secs(5 * 60); |
Contributor
Author
There was a problem hiding this comment.
This is a very generous debounce (and we re-jitter up to that, so we could wait up to 10 minutes before refreshing). The idea is to avoid spamming model refreshes on clients with patchy connections.
maxdeviant
approved these changes
May 27, 2026
TomPlanche
pushed a commit
to TomPlanche/zed
that referenced
this pull request
Jun 2, 2026
…7528) Until now, the cloud-hosted model list was only refreshed in response to events that exercise the LLM token (a `UserUpdated` push, an organization change, or `PrivateUserInfoUpdated`). If a user wasn't actively using AI features around the time we shipped new models, the list could stay stale until they restarted Zed. This is the second step toward fixing that, after zed-industries#57078 made the cloud websocket reconnect on its own. We now treat each successful (re)connect as a hint that the server state may have changed, so possibly new model definitions will be available, and trigger a model list refresh. The trigger is a new `Client::cloud_connection_id()` watch that bumps a counter each time the websocket handshake completes. `CloudLanguageModelProvider::State` subscribes to it and, on every tick after the initial `0`, schedules a debounced refresh (with jitter, so we don't have all active clients trying to reconnect at the same time after we deploy in cloud). Closes CLO-713. Release Notes: - The list of Zed hosted models is now refreshed automatically, without requiring a restart
This was referenced Jun 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Until now, the cloud-hosted model list was only refreshed in response to events that exercise the LLM token (a
UserUpdatedpush, an organization change, orPrivateUserInfoUpdated). If a user wasn't actively using AI features around the time we shipped new models, the list could stay stale until they restarted Zed.This is the second step toward fixing that, after #57078 made the cloud websocket reconnect on its own. We now treat each successful (re)connect as a hint that the server state may have changed, so possibly new model definitions will be available, and trigger a model list refresh.
The trigger is a new
Client::cloud_connection_id()watch that bumps a counter each time the websocket handshake completes.CloudLanguageModelProvider::Statesubscribes to it and, on every tick after the initial0, schedules a debounced refresh (with jitter, so we don't have all active clients trying to reconnect at the same time after we deploy in cloud).Closes CLO-713.
Release Notes: