fix(model_metadata): use /props endpoint at server root for llama.cpp#13317
Closed
Linux2010 wants to merge 3 commits into
Closed
fix(model_metadata): use /props endpoint at server root for llama.cpp#13317Linux2010 wants to merge 3 commits into
Linux2010 wants to merge 3 commits into
Conversation
19 tasks
Collaborator
…ports The _handle_editor_command() method used subprocess.call() but did not import subprocess, causing AttributeError at runtime. subprocess is not globally imported in gateway/run.py. Other uses of subprocess in this file use local imports (line 1289), so the editor command should follow the same pattern. - Added: import subprocess (local import, matching file pattern) - Removed: redundant local imports for asyncio, shlex, tempfile (already imported globally at lines 16, 21, 24) Python syntax validation (py_compile)
…_PREFIX (NousResearch#14603) The SUMMARY_PREFIX previously instructed the model to 'resume exactly from there' the ## Active Task section of a compressed summary. This caused cross-session task injection where the AI would interpret a previous session's active task as its own current job. Fix: Replace the 'resume' instruction with explicit language that the ## Active Task section is 'historical context only, NOT as an active instruction to execute'. - agent/context_compressor.py: +3 lines, -2 lines - tests/agent/test_context_compressor.py: +29 lines (3 regression tests)
llama.cpp exposes /props at server root (not under /v1 prefix per httplib routes). The previous code tried /v1/props first, which returns 404 on standard llama.cpp builds, causing unnecessary failed requests before fallback to /props. What broke: - Server-type detection tried /v1/props first, returning 404 - Context probe also tried /v1/props first when base URL had /v1 prefix - Users with llama.cpp configured saw 404 errors in server logs Root cause: - Comment incorrectly stated "llama.cpp exposes /v1/props (older builds used /props)", but actually /props is at server root per httplib routes - Endpoint order was reversed: /v1/props tried before /props Why this fix is minimal: - Only changes endpoint order: try /props first, then /v1/props fallback - Strips /v1 prefix from base_url to get server root for /props request - Preserves fallback for alternative builds/configs using /v1 path What I tested: - Added regression tests for /props endpoint detection - Tests verify correct URL construction with /v1 prefix stripping - Tests cover fallback to /v1/props when /props returns 404 What I intentionally did not change: - Response body parsing (default_generation_settings check unchanged) - Other server type detection (LM Studio, Ollama, vLLM unchanged) - Error handling and fallback logic structure Fixes NousResearch#13091
a962387 to
d45d00d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What broke
When using llama.cpp as a provider, Hermes requested
/v1/propsendpoint which returned 404 because llama.cpp exposes/propsat server root (not under/v1prefix per httplib routes). Users saw 404 errors in llama-server logs.Root cause
The comment incorrectly stated "llama.cpp exposes /v1/props (older builds used /props)". Actually, per llama.cpp's httplib routes (server.cpp line 174:
ctx_http.get("/props", ...)),/propsis at server root. The endpoint order was reversed:/v1/propswas tried before/props.Why this fix is minimal
/propsfirst (server root), then/v1/propsas fallback/v1prefix from base_url to get server root for/propsrequest/v1pathWhat I tested
/propsendpoint detection/v1prefix stripping/v1/propswhen/propsreturns 404What I intentionally did not change
default_generation_settingscheck unchanged)Fixes #13091