refactor: read config values dynamically instead of caching in instance variables by maobaolong · Pull Request #2610 · LMCache/LMCache

maobaolong · 2026-02-17T06:05:24Z

Remove cached instance variables for config values that should support dynamic configuration updates. Instead, read them directly from the config object at each usage site.

Configs changed to dynamic reading:

blocking_timeout_secs (remote_backend.py)
lookup_timeout_ms (lmcache_lookup_client.py, lmcache_async_lookup_client.py)
save_decode_cache (vllm_v1_adapter.py)
force_store_wait (cache_engine.py)
save_unfull_chunk (token_database.py)
hit_miss_ratio, chunk_size (hit_limit_lookup_client.py)
ping_timeout, get_blocking_failed_threshold, waiting_time_for_recovery (remote_backend_check.py)

What this PR does / why we need it:

Special notes for your reviewers:

If applicable:

this PR contains user facing changes - docs added
this PR contains unit tests

gemini-code-assist · 2026-02-17T06:05:47Z

Summary of Changes

Hello @maobaolong, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors several components to eliminate the caching of configuration values in instance variables. Instead, these values are now retrieved directly from the central configuration object whenever they are needed. This change is crucial for enabling dynamic updates to configuration settings without requiring application restarts or reinitialization of components, thereby improving the system's flexibility and responsiveness to operational changes.

Highlights

Dynamic Configuration Reading: Removed cached instance variables for various configuration values across multiple modules, ensuring they are read dynamically from the configuration object at each usage site.
Enhanced Flexibility: Enabled support for dynamic configuration updates for critical settings such as blocking_timeout_secs, lookup_timeout_ms, save_decode_cache, force_store_wait, save_unfull_chunk, hit_miss_ratio, chunk_size, ping_timeout, get_blocking_failed_threshold, and waiting_time_for_recovery.
Code Simplification: Streamlined code by removing redundant instance variables that previously stored static configuration values, promoting a more consistent approach to configuration access.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

lmcache/integration/vllm/vllm_v1_adapter.py
- Removed _save_decode_cache instance variable.
- Updated build_connector_meta to read save_decode_cache dynamically from self.config.
lmcache/v1/cache_engine.py
- Removed force_store_wait instance variable.
- Updated store and store_layer methods to read force_store_wait dynamically using self.config.get_extra_config_value.
lmcache/v1/health_monitor/checks/remote_backend_check.py
- Removed ping_timeout, get_blocking_failed_threshold, and waiting_time_for_recovery instance variables.
- Updated check method to dynamically retrieve waiting_time_for_recovery, get_blocking_failed_threshold, and ping_timeout from the backend config.
lmcache/v1/lookup_client/hit_limit_lookup_client.py
- Removed hit_ratio_upper and chunk_size instance variables.
- Updated lookup method to dynamically calculate hit_ratio_upper and retrieve chunk_size from self.config.
- Stored the config object as an instance variable self.config.
lmcache/v1/lookup_client/lmcache_async_lookup_client.py
- Removed timeout_ms instance variable.
- Updated lookup_cache method to read lookup_timeout_ms dynamically from self.config.
- Stored the config object as an instance variable self.config.
lmcache/v1/lookup_client/lmcache_lookup_client.py
- Removed timeout_ms instance variable.
- Updated socket options in __init__ and _recreate_socket to use self.config.lookup_timeout_ms directly.
lmcache/v1/storage_backend/remote_backend.py
- Removed blocking_timeout_secs instance variable.
- Updated get_blocking, batched_get_blocking, and async methods to read blocking_timeout_secs dynamically from self.config.
lmcache/v1/token_database.py
- Removed save_unfull_chunk instance variable.
- Updated _chunk_tokens method to read save_unfull_chunk dynamically from self.config.
- Stored the config object as an instance variable self.config.

Activity

No specific activity (comments, reviews, or progress updates) was provided in the context.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request refactors the code to read several configuration values dynamically from the config object instead of caching them as instance variables. This is a good change that allows for dynamic configuration updates. The implementation looks correct across all modified files, consistently replacing cached attributes with direct configuration access at the usage sites. I have one minor suggestion to improve maintainability by reducing code duplication in one of the files.

…ce variables Remove cached instance variables for config values that should support dynamic configuration updates. Instead, read them directly from the config object at each usage site. Configs changed to dynamic reading: - blocking_timeout_secs (remote_backend.py) - lookup_timeout_ms (lmcache_lookup_client.py, lmcache_async_lookup_client.py) - save_decode_cache (vllm_v1_adapter.py) - force_store_wait (cache_engine.py) - save_unfull_chunk (token_database.py) - hit_miss_ratio, chunk_size (hit_limit_lookup_client.py) - ping_timeout, get_blocking_failed_threshold, waiting_time_for_recovery (remote_backend_check.py) Signed-off-by: baoloongmao <baoloongmao@tencent.com>

Signed-off-by: baoloongmao <baoloongmao@tencent.com>

maobaolong · 2026-02-20T13:30:50Z

@sammshen @chunxiaozheng Would you like to take a look at this PR? Thanks!

Signed-off-by: baoloongmao <baoloongmao@tencent.com>

sammshen

LGTM

chunxiaozheng

lgtm

…ce variables (LMCache#2610) * refactor: read config values dynamically instead of caching in instance variables Remove cached instance variables for config values that should support dynamic configuration updates. Instead, read them directly from the config object at each usage site. Configs changed to dynamic reading: - blocking_timeout_secs (remote_backend.py) - lookup_timeout_ms (lmcache_lookup_client.py, lmcache_async_lookup_client.py) - save_decode_cache (vllm_v1_adapter.py) - force_store_wait (cache_engine.py) - save_unfull_chunk (token_database.py) - hit_miss_ratio, chunk_size (hit_limit_lookup_client.py) - ping_timeout, get_blocking_failed_threshold, waiting_time_for_recovery (remote_backend_check.py) Signed-off-by: baoloongmao <baoloongmao@tencent.com> * Address gemini's comment Signed-off-by: baoloongmao <baoloongmao@tencent.com> * Fix code quality issue Signed-off-by: baoloongmao <baoloongmao@tencent.com> --------- Signed-off-by: baoloongmao <baoloongmao@tencent.com>

…ce variables (LMCache#2610) * refactor: read config values dynamically instead of caching in instance variables Remove cached instance variables for config values that should support dynamic configuration updates. Instead, read them directly from the config object at each usage site. Configs changed to dynamic reading: - blocking_timeout_secs (remote_backend.py) - lookup_timeout_ms (lmcache_lookup_client.py, lmcache_async_lookup_client.py) - save_decode_cache (vllm_v1_adapter.py) - force_store_wait (cache_engine.py) - save_unfull_chunk (token_database.py) - hit_miss_ratio, chunk_size (hit_limit_lookup_client.py) - ping_timeout, get_blocking_failed_threshold, waiting_time_for_recovery (remote_backend_check.py) Signed-off-by: baoloongmao <baoloongmao@tencent.com> * Address gemini's comment Signed-off-by: baoloongmao <baoloongmao@tencent.com> * Fix code quality issue Signed-off-by: baoloongmao <baoloongmao@tencent.com> --------- Signed-off-by: baoloongmao <baoloongmao@tencent.com> Signed-off-by: shaoxiawjc <wjc2800@163.com>

…ce variables (LMCache#2610) * refactor: read config values dynamically instead of caching in instance variables Remove cached instance variables for config values that should support dynamic configuration updates. Instead, read them directly from the config object at each usage site. Configs changed to dynamic reading: - blocking_timeout_secs (remote_backend.py) - lookup_timeout_ms (lmcache_lookup_client.py, lmcache_async_lookup_client.py) - save_decode_cache (vllm_v1_adapter.py) - force_store_wait (cache_engine.py) - save_unfull_chunk (token_database.py) - hit_miss_ratio, chunk_size (hit_limit_lookup_client.py) - ping_timeout, get_blocking_failed_threshold, waiting_time_for_recovery (remote_backend_check.py) Signed-off-by: baoloongmao <baoloongmao@tencent.com> * Address gemini's comment Signed-off-by: baoloongmao <baoloongmao@tencent.com> * Fix code quality issue Signed-off-by: baoloongmao <baoloongmao@tencent.com> --------- Signed-off-by: baoloongmao <baoloongmao@tencent.com> Signed-off-by: Aaron Wu <aaron.wu@dell.com>

maobaolong requested a review from sammshen February 17, 2026 06:05

maobaolong force-pushed the updateConfSupport branch from fcedae1 to 8d45048 Compare February 17, 2026 06:11

gemini-code-assist Bot reviewed Feb 17, 2026

View reviewed changes

Comment thread lmcache/v1/health_monitor/checks/remote_backend_check.py Outdated

maobaolong added the full Run comprehensive tests on this PR label Feb 19, 2026

maobaolong added 2 commits February 19, 2026 21:12

Address gemini's comment

d6f364c

Signed-off-by: baoloongmao <baoloongmao@tencent.com>

maobaolong force-pushed the updateConfSupport branch from 086b3fb to d6f364c Compare February 20, 2026 13:29

Fix code quality issue

c64e3ee

Signed-off-by: baoloongmao <baoloongmao@tencent.com>

sammshen approved these changes Feb 22, 2026

View reviewed changes

chunxiaozheng approved these changes Feb 24, 2026

View reviewed changes

chunxiaozheng merged commit 3cbd2b7 into LMCache:dev Feb 24, 2026
39 of 46 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: read config values dynamically instead of caching in instance variables#2610

refactor: read config values dynamically instead of caching in instance variables#2610
chunxiaozheng merged 3 commits intoLMCache:devfrom
maobaolong:updateConfSupport

maobaolong commented Feb 17, 2026

Uh oh!

gemini-code-assist Bot commented Feb 17, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

maobaolong commented Feb 20, 2026

Uh oh!

sammshen left a comment

Uh oh!

chunxiaozheng left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

maobaolong commented Feb 17, 2026

Uh oh!

gemini-code-assist Bot commented Feb 17, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

maobaolong commented Feb 20, 2026

Uh oh!

sammshen left a comment

Choose a reason for hiding this comment

Uh oh!

chunxiaozheng left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants