Skip to content

inst : preview generation + improve prompt#98

Merged
ggerganov merged 15 commits into
masterfrom
gg/inst-cont
Jan 24, 2026
Merged

inst : preview generation + improve prompt#98
ggerganov merged 15 commits into
masterfrom
gg/inst-cont

Conversation

@ggerganov

@ggerganov ggerganov commented Jan 20, 2026

Copy link
Copy Markdown
Member

cont #96

  • Streaming response
  • Better system prompt
  • Continue instructions (<leader>llc)
  • Rerun instructions (<leader>llr)
  • Update default keymaps - use <leader>ll as common keymap

Next PRs:

  • Multiple suggestions using parallel n_cmpl
  • Auto-trigger instructions based on git diff?
llama.vim-inst-1-lq.mp4

@alopatindev

alopatindev commented Jan 22, 2026

Copy link
Copy Markdown
let l:system_prompt .= "... Respond ONLY with the result of applying INSTRUCTION to SELECTION given the CONTEXT. .... Do not output any extra separators.\n"

let l:extra = s:ring_get_extra()

let l:system_prompt .= "\n"
let l:system_prompt .= "--- CONTEXT   --------------------------------------------------\n"
...

Looks kinda sad. Maybe at least serialize inputs as JSONs?

This for instance works okay with ollama most of the time:

curl -s http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen2.5-coder:7b",
    "prompt": "You are a code-editing assistant that receives JSONs with \"selection\" value as input. Apply \"instruction\" value to the input, answer ONLY with single VALID minified JSON object with the ONLY value \"replacement\" that contains ONLY the output code. No markdown wrapping. {\"selection\":\"// TODO\",\"instruction\":\"implement quicksort in C\"}",
    "stream": false
  }' | jq -rM '.response' | jq -rM '.replacement'

@ggerganov

Copy link
Copy Markdown
Member Author

Yeah, I was planning to change to json. Btw, I'm using gpt-oss-120b and it's solid.

@ggerganov

Copy link
Copy Markdown
Member Author

@alopatindev Should be improved now - I find this feature quite useful.

If you give it a try would appreciate feedback.

@alopatindev

alopatindev commented Jan 24, 2026

Copy link
Copy Markdown

Works for me (nvim 0.11.5), thanks!

Tested with --fim-qwen-3b-default and --fim-qwen-7b-default. I don't have enough VRAM to test with larger models.

@ggerganov ggerganov merged commit 4cdf1dd into master Jan 24, 2026
@ggerganov ggerganov deleted the gg/inst-cont branch January 24, 2026 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants