Allow changing the context window size for Ollama#44506
Allow changing the context window size for Ollama#44506ConradIrwin merged 3 commits intozed-industries:mainfrom
Conversation
1ba3d49 to
5811ddc
Compare
|
Can you tell me about the motivation for this change and what alternatives there may have been? |
For context I use zed with ollama with 48gb vram. It is also known most local models do not operate well on consumer hardware at big context window so leaving this to be randomly decided by Zed based on vram availability and model config is a bad combination. It is easier to setup a context window at the provider level like is done on other IDE addons such as cline, continue, roo, kilo, etc. So all models will use such context. |
|
Great! Could you please change the name of the variable in code from Also, would you mind explaining the change to the |
d9843c0 to
e3debc4
Compare
|
I don't think renaming context window to max tokens is a good idea. As token limits can be imposed on outputs making "Max Tokens" misleading. The get_max_tokens function now only acts as fallback if the max_tokens var is passed empty under any user input error such as " " |
e3debc4 to
dff884a
Compare
|
No progress on this? |
|
@sbe-arg Sorry for the dropped hand-offs here. Thanks for tidying this up. |
Auto-applied queued documentation suggestions from: - PR #48908 - PR #48909 - PR #48910 - PR #48912 - PR #48930 - PR #44794 - PR #48763 - PR #45073 - PR #48495 - PR #49374 - PR #49139 - PR #48780 - PR #48619 - PR #48978 - PR #48962 - PR #48988 - PR #47860 - PR #49015 - PR #47095 - PR #47475 - PR #48542 - PR #46766 - PR #47754 - PR #48807 - PR #44506 - PR #49051 - PR #49069 - PR #48842 - PR #48851 - PR #48736 - PR #47673 - PR #49094 - PR #49098 - PR #49622 Generated with script/docs-suggest-publish for human review in draft PR.
Auto-applied queued documentation suggestions from: - PR #48908 - PR #48909 - PR #48910 - PR #48912 - PR #48930 - PR #44794 - PR #48763 - PR #45073 - PR #48495 - PR #49374 - PR #49139 - PR #48780 - PR #48619 - PR #48978 - PR #48962 - PR #48988 - PR #47860 - PR #49015 - PR #47095 - PR #47475 - PR #48542 - PR #46766 - PR #47754 - PR #48807 - PR #44506 - PR #49051 - PR #49069 - PR #48842 - PR #48851 - PR #48736 - PR #47673 - PR #49094 - PR #49098 - PR #49622 - PR #49554 - PR #49710 - PR #49716 - PR #49732 - PR #49788 - PR #49876 - PR #49902 - PR #49910 - PR #49390 - PR #50027 Generated with script/docs-suggest-publish for human review in draft PR.
Auto-applied documentation from: - PR #48542: Bedrock extended context window - PR #46766: LSP adapters in settings schema - PR #47754: VSCode tasks.json label generation Skipped (no target file exists): - PR #49069: panel zoom state persistence Already documented from prior batches: - PR #48807, PR #44506, PR #49051, PR #48842, PR #48851, PR #48736
Auto-applied queued documentation suggestions from: - PR #48908 - PR #48909 - PR #48910 - PR #48912 - PR #48930 - PR #44794 - PR #48763 - PR #45073 - PR #48495 - PR #49374 - PR #49139 - PR #48780 - PR #48619 - PR #48978 - PR #48962 - PR #48988 - PR #47860 - PR #49015 - PR #47095 - PR #47475 - PR #48542 - PR #46766 - PR #47754 - PR #48807 - PR #44506 - PR #49051 - PR #49069 - PR #48842 - PR #48851 - PR #48736 - PR #47673 - PR #49094 - PR #49098 - PR #49622 - PR #49554 - PR #49710 - PR #49716 - PR #49732 - PR #49788 - PR #49876 - PR #49902 - PR #49910 - PR #49390 - PR #50027 Generated with script/docs-suggest-publish for human review in draft PR.
Auto-applied documentation from: - PR #48542: Bedrock extended context window - PR #46766: LSP adapters in settings schema - PR #47754: VSCode tasks.json label generation Skipped (no target file exists): - PR #49069: panel zoom state persistence Already documented from prior batches: - PR #48807, PR #44506, PR #49051, PR #48842, PR #48851, PR #48736
Release Notes: