🔒 Aisle Security Analysis

We found 1 potential security issue(s) in this PR:

1. 🔵 Unbounded maxTokens derived from unauthenticated Venice /models response (cost/DoS amplification)

Description

discoverVeniceModels() trusts the public (unauthenticated) Venice /models response field model_spec.maxCompletionTokens and maps it into internal ModelDefinitionConfig.maxTokens without applying a sane upper bound.

Implications if maxTokens is used downstream as a default/limit for generation:

• A malicious/compromised /models response (or network manipulation in less-trusted environments) can set maxCompletionTokens to an extremely large value.
• This value is persisted into generated models.json and then used to construct runtime models, potentially enabling very large generations (unexpected billing) and/or resource exhaustion (retries, large responses, long-running streams).

Vulnerable code:

function resolveApiMaxCompletionTokens(apiModel: VeniceModel): number | undefined {
  const raw = apiModel.model_spec.maxCompletionTokens;
  if (typeof raw === "number" && Number.isFinite(raw) && raw > 0) {
    return raw;
  }
  return undefined;
}
...
const apiMaxTokens = resolveApiMaxCompletionTokens(apiModel);
...
if (apiMaxTokens !== undefined) {
  def.maxTokens = apiMaxTokens;
}
...
maxTokens: apiMaxTokens ?? VENICE_DEFAULT_MAX_TOKENS,

Note: elsewhere in the codebase, some tools clamp their own maxTokens request options, but the core model object (used for agent runs) is still populated from discovery results and can flow into provider calls via the underlying SDK.

Recommendation

Clamp and normalize the API-provided value before using it.

Guidelines:

• Require an integer (floor/round) and reject unreasonable values.
• Cap by contextWindow (and/or Venice-reported availableContextTokens).
• Apply a global/provider-specific hard maximum (e.g. 8192/16384), unless you explicitly support larger outputs.

Example fix:

const VENICE_HARD_MAX_COMPLETION_TOKENS = 8192; // or provider-specific

function resolveApiMaxCompletionTokens(apiModel: VeniceModel): number | undefined {
  const raw = apiModel.model_spec.maxCompletionTokens;
  if (typeof raw !== "number" || !Number.isFinite(raw) || raw <= 0) {
    return undefined;
  }

  const available = apiModel.model_spec.availableContextTokens;
  const capByContext =
    typeof available === "number" && Number.isFinite(available) && available > 0
      ? available
      : VENICE_HARD_MAX_COMPLETION_TOKENS;

  const normalized = Math.floor(raw);
  return Math.min(normalized, capByContext, VENICE_HARD_MAX_COMPLETION_TOKENS);
}

Additionally consider:

• For catalog models, clamp by the catalog entry’s known maxTokens (or a safe maximum) rather than letting the network response override upward.
• If Venice discovery is purely informational, treat the response as untrusted and do not use it to raise limits.

Analyzed PR: #38182 at commit a7cb9c5

_{Last updated on: 2026-03-06T17:19:28Z}

Greptile Summary

This PR fixes a real production-breaking bug where all Venice models used a hardcoded maxTokens: 8192, causing HTTP 400 errors on models that only allow up to 4096 completion tokens. The fix is well-scoped: it reads maxCompletionTokens from the Venice /models API response, applies it via a clean validation helper, lowers the static catalog defaults for the affected models, and adds a safe fallback for unknown models.

• Core logic in venice-models.ts is correct — the resolveApiMaxCompletionTokens guard properly rejects invalid values (NaN, Infinity, ≤0) and the discovery loop correctly prefers API values over catalog values when present, and catalog values over the global default when the API omits the field.
• Key test gap: the "uses API maxCompletionTokens for catalog models when present" test uses maxCompletionTokens: 4096, which matches the catalog's existing 4096 for llama-3.3-70b. The test passes vacuously and would not catch a regression that removed the override entirely. A distinct value (e.g. 2048) should be used.
• Missing coverage: no test asserts that a catalog model with maxTokens: 8192 (e.g. qwen3-235b-a22b-instruct-2507) retains its catalog value when the API omits maxCompletionTokens, preventing a future regression that silently caps it to VENICE_DEFAULT_MAX_TOKENS (4096).

Confidence Score: 4/5

• Safe to merge — the production fix is correct; two test quality issues should be addressed but do not affect runtime behavior.
• The implementation logic is sound and correctly resolves the 400 error for affected models. The one point deduction is for the vacuous catalog-override test, which provides false confidence that the API override path works when it actually tests a no-op (same value in API and catalog). This doesn't represent a runtime bug but does mean a regression in that code path could go undetected.
• src/agents/venice-models.test.ts — two test cases need attention as described in the review comments.

_{Last reviewed commit: a7cb9c5}

Nice work surfacing the safer per-model override path here.

I'm consolidating this Venice bug cluster under #38168 before we merge anything, so I'm keeping the issue as the single canonical thread and marking overlapping PR attempts as duplicates for hygiene. This branch is one of the strongest partial fixes in the set, and I expect to reuse the good parts rather than let a handful of similar PRs compete.

Your contribution is preserved in that credit trail. If you think this branch covers something outside #38168, point me at it and I'll re-check the split.