fix(proxy): load REDIS_* env vars when cache_params has non-connection keys by dschulmeist · Pull Request #26244 · BerriAI/litellm

dschulmeist · 2026-04-22T12:18:14Z

Relevant issues

Type

Bug Fix

Changes

Problem

In ProxyConfig.load_config, the Redis env-var fallback is gated on len(cache_params.keys()) == 0:

if (cache_type == "redis" or cache_type == "redis-semantic") \
    and len(cache_params.keys()) == 0:
    # load REDIS_HOST / REDIS_PORT / REDIS_PASSWORD

This means a config like:

litellm_settings:
  cache: true
  cache_params:
    mode: default_off

combined with REDIS_HOST + REDIS_PORT env vars silently falls back to InMemoryCache — because the presence of mode makes cache_params non-empty, skipping the env-var branch entirely.

Downstream, spend_counter_cache.redis_cache stays None and per-pod counters diverge. In multi-pod deployments this breaks spend tracking and budget enforcement without any error.

Fix

Replace the length check with "no connection details supplied":

if (cache_type == "redis" or cache_type == "redis-semantic") and (
    "host" not in cache_params and "url" not in cache_params
):

The env-var fallback now triggers whenever the user has not specified host or url. Other cache_params keys (mode, ttl, default_in_memory_ttl, etc.) no longer suppress it.

Tests

Two new tests in tests/test_litellm/proxy/test_proxy_server.py:

test_redis_env_vars_loaded_when_cache_params_has_non_connection_keys — regression test for the reported bug.
test_explicit_cache_params_host_not_overwritten_by_env_vars — ensures env-var fallback does not clobber user-supplied host.

Both patch ProxyConfig._init_cache and assert the computed cache_params dict contains the expected Redis connection values.

Pre-Submission checklist

Added testing in tests/test_litellm/
PR passes make test-unit on the touched file (110/110 in test_proxy_server.py)
Scope isolated to a single bug
Greptile review (will request after submission)

CLAassistant · 2026-04-22T12:18:20Z

All committers have signed the CLA.

gitguardian · 2026-04-22T12:18:49Z

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
While these secrets were previously flagged, we no longer have a reference to the
specific commits where they were detected. Once a secret has been leaked into a git
repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.

^{_{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}}

greptile-apps · 2026-04-22T12:21:42Z

Greptile Summary

This PR fixes a silent fallback to InMemoryCache when cache_params contained any non-connection key (e.g. mode: default_off). The old guard len(cache_params.keys()) == 0 was too strict; the fix replaces it with "host" not in cache_params and "url" not in cache_params, so the REDIS_* env-var fallback fires whenever no connection details are explicitly supplied. Two focused regression tests are added and no existing tests are modified.

Confidence Score: 5/5

Safe to merge — minimal, well-tested fix that restores expected Redis env-var fallback behaviour.

The one-line logic change is correct and tightly scoped. Both new tests are mock-only, cover the exact regression and its guard condition, and no existing tests were modified. No security, backwards-compatibility, or data-integrity concerns are introduced.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/proxy_server.py	Replaced empty-dict guard with connection-key check; env-var fallback now correctly fires when only non-connection keys (e.g. `mode`) are present in `cache_params`.
tests/test_litellm/proxy/test_proxy_server.py	Two new async regression tests added: one verifying env-var fallback with non-connection keys present, one verifying explicit host is not overwritten. Both use mocks only — no real network calls.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[load_config: cache=true] --> B{cache_params has\n'host' or 'url'?}
    B -- Yes --> C[Use explicit connection\ndetails from cache_params]
    B -- No --> D[Fall back to\nREDIS_HOST / REDIS_PORT /\nREDIS_PASSWORD env vars]
    D --> E[cache_params.update with\nenv-var values]
    C --> F[_init_cache cache_params]
    E --> F
    F --> G[Redis / redis-semantic cache\ninitialized correctly]

    style D fill:#c8f7c5
    style B fill:#fff3cd

_{Reviews (3): Last reviewed commit: "fix(proxy): load REDIS_* env vars when c..." | Re-trigger Greptile}

codspeed-hq · 2026-04-22T12:22:32Z

Congrats! CodSpeed is installed 🎉

🆕 16 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

test_get_model_cost_key_case_insensitive: 81.7 µs
test_token_counter_multi_turn: 562 µs
test_cost_per_token_anthropic: 540.6 µs
test_get_model_info_with_provider: 79.6 µs
test_cost_per_token_openai: 541.2 µs
test_get_model_info_anthropic: 78.4 µs
test_token_counter_with_tools: 413.7 µs
test_get_llm_provider_with_prefix: 136.5 µs
test_token_counter_simple_message: 235.2 µs
test_token_counter_long_content: 1.7 ms
test_get_llm_provider_openai: 137.6 µs
test_get_model_info_openai: 79.9 µs
test_get_model_cost_key_exact_match: 79.3 µs
test_token_counter_raw_text: 183.3 µs
test_get_llm_provider_anthropic: 140.9 µs
test_get_llm_provider_azure: 143.5 µs

…n keys (BerriAI#26233) The cache_params env-var fallback in ProxyConfig.load_config was gated on `len(cache_params.keys()) == 0`, so any non-empty cache_params (e.g. just `mode: default_off`) silently dropped the Redis env var config and fell back to in-memory cache. This broke multi-pod deployments because spend_counter_cache.redis_cache never got wired up and each pod tracked counters independently. Replace the length check with a "user did not supply connection details" check: only populate REDIS_HOST / REDIS_PORT / REDIS_PASSWORD from the environment when neither `host` nor `url` is present in cache_params. Other cache_params keys (mode, ttl, etc.) no longer suppress the fallback. Adds two regression tests exercising ProxyConfig.load_config: - cache_params with only non-connection keys → env vars load, RedisCache path - cache_params with explicit host → env vars do not overwrite user config Fixes BerriAI#26233

dschulmeist force-pushed the fix/redis-env-vars-cache-params-26233 branch from 9f02a75 to fdcd0d7 Compare April 22, 2026 12:19

dschulmeist force-pushed the fix/redis-env-vars-cache-params-26233 branch 2 times, most recently from 00f47fa to 1372e07 Compare April 22, 2026 12:22

dschulmeist changed the base branch from main to litellm_oss_branch April 22, 2026 12:23

dschulmeist force-pushed the fix/redis-env-vars-cache-params-26233 branch from 1372e07 to 22a3c50 Compare April 22, 2026 12:24

Sameerlite deleted the branch BerriAI:litellm_oss_branch April 27, 2026 04:56

Sameerlite closed this Apr 27, 2026

dschulmeist mentioned this pull request Apr 27, 2026

fix(proxy): load REDIS_* env vars when cache_params has non-connection keys #26576

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(proxy): load REDIS_* env vars when cache_params has non-connection keys#26244

fix(proxy): load REDIS_* env vars when cache_params has non-connection keys#26244
dschulmeist wants to merge 1 commit into
BerriAI:litellm_oss_branchfrom
dschulmeist:fix/redis-env-vars-cache-params-26233

dschulmeist commented Apr 22, 2026

Uh oh!

CLAassistant commented Apr 22, 2026 •

edited

Loading

Uh oh!

gitguardian Bot commented Apr 22, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Apr 22, 2026 •

edited

Loading

Important Files Changed

Uh oh!

codspeed-hq Bot commented Apr 22, 2026 •

edited

Loading

Detected benchmarks

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

dschulmeist commented Apr 22, 2026

Relevant issues

Type

Changes

Problem

Fix

Tests

Pre-Submission checklist

Uh oh!

CLAassistant commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gitguardian Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

️✅ There are no secrets present in this pull request anymore.

Uh oh!

greptile-apps Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

codspeed-hq Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Congrats! CodSpeed is installed 🎉

Detected benchmarks

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

CLAassistant commented Apr 22, 2026 •

edited

Loading

gitguardian Bot commented Apr 22, 2026 •

edited

Loading

greptile-apps Bot commented Apr 22, 2026 •

edited

Loading

codspeed-hq Bot commented Apr 22, 2026 •

edited

Loading