Skip to content

edit_prediction: Support self-hosted Sweep Next Edit models#51139

Open
avoidthekitchen wants to merge 5 commits intozed-industries:mainfrom
avoidthekitchen:codex-sweep-prompt-format
Open

edit_prediction: Support self-hosted Sweep Next Edit models#51139
avoidthekitchen wants to merge 5 commits intozed-industries:mainfrom
avoidthekitchen:codex-sweep-prompt-format

Conversation

@avoidthekitchen
Copy link
Copy Markdown

@avoidthekitchen avoidthekitchen commented Mar 9, 2026

This adds support for running self-hosted Sweep Next Edit edit prediction models through Zed's OpenAI-compatible provider. The target use case is a local deployment based on sweepai/sweep-next-edit-1.5B, using the rewrite-window prompting approach described in OSS Next Edit. Personal user testing shows impressive quality from this small edit prediction model.

This is valuable because it allows more flexibility for Zed users to self-host state of the art next edit prediction models!

This PR addresses the self-hosted workflow discussed in #50929.
Note that the self-hosted model path needed more than just a new dropdown value. Zed has to build the rewrite-window prompt shape the model expects, map the rewritten window back into anchored edits, and make prompt_format: "infer" work with the filename-style model identifiers that llama-server local server reports.

The main changes are:

  • add a Sweep prompt format for OpenAI-compatible edit prediction providers and route it to a dedicated SweepPrompt model path
  • build Sweep rewrite-window prompts from the active cursor window, recent change history, and related file excerpts
  • convert rewrite responses back into anchored edits and suppress no-op rewrites
  • add fixed cursor-window extraction used by the Sweep rewrite prompt path
  • recognize sweep-next-edit model names during infer, including filename-style identifiers such as sweepai_sweep-next-edit-1.5B_sweep-next-edit-1.5b.q8_0.v2.gguf
  • reject reserved Sweep prompt tokens before sending malformed requests to a self-hosted server
  • update the docs to describe the actual Sweep rewrite-window prompt format

Local setup used for manual verification:

llama-server \
  --hf-repo sweepai/sweep-next-edit-1.5B \
  --hf-file sweep-next-edit-1.5b.q8_0.v2.gguf \
  --ctx-size 8192 \
  --host 127.0.0.1 \
  --port 8080
{
  "edit_predictions": {
    "provider": "open_ai_compatible_api",
    "open_ai_compatible_api": {
      "api_url": "http://127.0.0.1:8080/v1/completions",
      "model": "sweepai_sweep-next-edit-1.5B_sweep-next-edit-1.5b.q8_0.v2.gguf",
      "prompt_format": "infer",
      "max_output_tokens": 512
    }
  }
}

The request that produced this PR included screenshots of the OpenAI-compatible provider configuration and the prompt format dropdown with Sweep selected.

Tests added in this branch:

  • test_sweep_prompt_format_routes_to_sweep_prompt_model
  • test_fixed_line_window_around_cursor_start_middle_and_end
  • test_sweep_prompt_request_prediction_diffs_rewritten_window_into_anchored_edits
  • test_sweep_prompt_request_prediction_returns_none_for_identical_rewrite
  • test_original_window_for_current_window_uses_latest_pre_edit_snapshot
  • test_original_window_for_current_window_returns_none_without_matching_history
  • test_recent_change_block_from_event_formats_original_and_updated_sections

Verification run locally:

  • cargo test -p zed sweep_prompt_format_routes
  • cargo test -p zed subscribe_uses_stale_provider_config
  • cargo test -p edit_prediction sweep_prompt
  • cargo test -p edit_prediction fixed_line_window_around_cursor_start_middle_and_end
  • cargo test -p edit_prediction test_sweep_prompt_request_prediction_diffs_rewritten_window_into_anchored_edits
  • cargo test -p edit_prediction test_sweep_prompt_request_prediction_returns_none_for_identical_rewrite
  • ./script/clippy -p zed -p edit_prediction -p settings_content

Release Notes:

  • Added support for self-hosted Sweep Next Edit models in OpenAI-compatible edit predictions, including the sweep prompt format and infer detection for sweep-next-edit model names.

@cla-bot
Copy link
Copy Markdown

cla-bot bot commented Mar 9, 2026

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: mistercheese.
This is most likely caused by a git client misconfiguration; please make sure to:

  1. check if your git client is configured with an email to sign commits git config --list | grep email
  2. If not, set it up using git config --global user.email email@example.com
  3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

@zed-community-bot zed-community-bot bot added the first contribution the author's first pull request to Zed. NOTE: the label application is automated via github actions label Mar 9, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 497eccb7c0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@avoidthekitchen avoidthekitchen force-pushed the codex-sweep-prompt-format branch from 497eccb to 23a28d3 Compare March 9, 2026 23:53
@cla-bot cla-bot bot added the cla-signed The user has signed the Contributor License Agreement label Mar 9, 2026
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@avoidthekitchen avoidthekitchen force-pushed the codex-sweep-prompt-format branch from 23a28d3 to ab90530 Compare March 10, 2026 00:11
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@SomeoneToIgnore SomeoneToIgnore added the area:ai Related to Agent Panel, Edit Prediction, Copilot, or other AI features label Mar 12, 2026
@guiopen guiopen mentioned this pull request Apr 1, 2026
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:ai Related to Agent Panel, Edit Prediction, Copilot, or other AI features cla-signed The user has signed the Contributor License Agreement first contribution the author's first pull request to Zed. NOTE: the label application is automated via github actions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants