Skip to content

qmd query fails with large files: reranker context size exceeded #91

@AustinBrister

Description

@AustinBrister

Description

qmd query fails when any of the top 30 candidate documents is large (e.g., 2MB+). The error:

The input lengths of some of the given documents exceed the context size. Try to increase the context size to at least 93075 or use another model that supports longer contexts.

Root Cause

In src/mcp.ts lines 407-410, the full document body is passed to the reranker:

const reranked = await store.rerank(
  query,
  candidates.map(c => ({ file: c.file, text: c.body })),  // FULL BODY
  DEFAULT_RERANK_MODEL
);

When a candidate has a 2.4MB body (~600K tokens), it exceeds the qwen3-reranker-0.6b model's context window.

Reproduction

  1. Index a collection containing large markdown files (e.g., full TRCP/FRCP appendices at 1-2MB each)
  2. Run any qmd query that returns one of these files as a candidate
  3. Observe context size error

Suggested Fix

Truncate document bodies before passing to reranker:

candidates.map(c => ({ file: c.file, text: c.body.slice(0, 8000) }))

Or better: use the matched snippet/chunk rather than full body, since the reranker only needs enough context to judge relevance.

Environment

  • qmd 1.0.0
  • macOS (Apple Silicon)
  • Collection: 8000+ markdown files including some 1-2MB appendix files

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions