edit_prediction: Support self-hosted Sweep Next Edit models#51139
edit_prediction: Support self-hosted Sweep Next Edit models#51139avoidthekitchen wants to merge 5 commits intozed-industries:mainfrom
Conversation
|
Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: mistercheese.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 497eccb7c0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
497eccb to
23a28d3
Compare
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
23a28d3 to
ab90530
Compare
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This adds support for running self-hosted Sweep Next Edit edit prediction models through Zed's OpenAI-compatible provider. The target use case is a local deployment based on sweepai/sweep-next-edit-1.5B, using the rewrite-window prompting approach described in OSS Next Edit. Personal user testing shows impressive quality from this small edit prediction model.
This is valuable because it allows more flexibility for Zed users to self-host state of the art next edit prediction models!
This PR addresses the self-hosted workflow discussed in #50929.
Note that the self-hosted model path needed more than just a new dropdown value. Zed has to build the rewrite-window prompt shape the model expects, map the rewritten window back into anchored edits, and make
prompt_format: "infer"work with the filename-style model identifiers that llama-server local server reports.The main changes are:
Sweepprompt format for OpenAI-compatible edit prediction providers and route it to a dedicatedSweepPromptmodel pathsweep-next-editmodel names duringinfer, including filename-style identifiers such assweepai_sweep-next-edit-1.5B_sweep-next-edit-1.5b.q8_0.v2.ggufLocal setup used for manual verification:
{ "edit_predictions": { "provider": "open_ai_compatible_api", "open_ai_compatible_api": { "api_url": "http://127.0.0.1:8080/v1/completions", "model": "sweepai_sweep-next-edit-1.5B_sweep-next-edit-1.5b.q8_0.v2.gguf", "prompt_format": "infer", "max_output_tokens": 512 } } }The request that produced this PR included screenshots of the OpenAI-compatible provider configuration and the prompt format dropdown with
Sweepselected.Tests added in this branch:
test_sweep_prompt_format_routes_to_sweep_prompt_modeltest_fixed_line_window_around_cursor_start_middle_and_endtest_sweep_prompt_request_prediction_diffs_rewritten_window_into_anchored_editstest_sweep_prompt_request_prediction_returns_none_for_identical_rewritetest_original_window_for_current_window_uses_latest_pre_edit_snapshottest_original_window_for_current_window_returns_none_without_matching_historytest_recent_change_block_from_event_formats_original_and_updated_sectionsVerification run locally:
cargo test -p zed sweep_prompt_format_routescargo test -p zed subscribe_uses_stale_provider_configcargo test -p edit_prediction sweep_promptcargo test -p edit_prediction fixed_line_window_around_cursor_start_middle_and_endcargo test -p edit_prediction test_sweep_prompt_request_prediction_diffs_rewritten_window_into_anchored_editscargo test -p edit_prediction test_sweep_prompt_request_prediction_returns_none_for_identical_rewrite./script/clippy -p zed -p edit_prediction -p settings_contentRelease Notes:
sweepprompt format andinferdetection forsweep-next-editmodel names.