Skip to content

[Bugfix] fix the concurrency bug during non-blocking store()#12

Closed
ApostaC wants to merge 6 commits intoLMCache:mainfrom
ApostaC:dev/async-put-bugfix
Closed

[Bugfix] fix the concurrency bug during non-blocking store()#12
ApostaC wants to merge 6 commits intoLMCache:mainfrom
ApostaC:dev/async-put-bugfix

Conversation

@ApostaC
Copy link
Copy Markdown
Contributor

@ApostaC ApostaC commented Jul 5, 2024

No description provided.

@ApostaC ApostaC requested a review from YaoJiayi July 5, 2024 01:59
@ApostaC ApostaC closed this Jul 5, 2024
njhill pushed a commit to njhill/LMCache that referenced this pull request May 13, 2025
sheperdh added a commit to sheperdh/LMCache2 that referenced this pull request Jan 6, 2026
* [XWKV-48] Implement gismo backend (LMCache#7)

* [XWKV-48] Cache fd to optimize performance (LMCache#11)

Cache fd to optimize performance.

Add thread pool to read/write file in parallel.

* [XWKV-48] Support remotely read kv (LMCache#12)

As Gismo supports locally read files
written remotely, we need to change our code
to make sure we can read files not cached.

* [XWKV-68] Use vram API to boost performance (LMCache#13)

As vram read/write API to boost performance.

Use new get/put method to read/write meta file.

* [XWKV-68] Use batched contains API

Use mvfs batched contains API to implement
batch contains interface to
boost performance.

Add retry when get metadata in case lmcache
reads faster than backend.

---------

Co-authored-by: Jinwen <287310886@qq.com>
yoo-kumaneko added a commit to yoo-kumaneko/LMCache that referenced this pull request Apr 9, 2026
…e#12)

* feat: cherry-pick chunk hash file logger from PR LMCache#2928

Cherry-pick squashed changes from LMCache#2928 which adds
a chunk hash file logger to the MP server for offline analysis.

Signed-off-by: root <crclq2018@gmail.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: rigginschen <rigginschen@tencent.com>

* feat: add chunk_size, seq_len, dtypes, chunk_byte_size to hash log

Add per-entry metadata to chunk hash JSONL output: chunk_size
(tokens per chunk), seq_len (request sequence length), dtypes
(KV cache data types), and chunk_byte_size (bytes per chunk).

Signed-off-by: root <crclq2018@gmail.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: rigginschen <rigginschen@tencent.com>

---------

Signed-off-by: rigginschen <rigginschen@tencent.com>
Co-authored-by: rigginschen <rigginschen@tencent.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
yoo-kumaneko pushed a commit to yoo-kumaneko/LMCache that referenced this pull request Apr 13, 2026
…LMCache#12)"

This reverts commit ee037db.

Signed-off-by: rigginschen <rigginschen@tencent.com>
yoo-kumaneko added a commit to yoo-kumaneko/LMCache that referenced this pull request Apr 14, 2026
* Revert "feat: cherry-pick chunk hash file logger from PR LMCache#2928 (LMCache#12)"

This reverts commit ee037db.

Signed-off-by: rigginschen <rigginschen@tencent.com>

* feat: add chunk hash logger as EventBus subscriber

Add JSONL-based chunk hash logging to the multiprocess server for
offline analysis of KV cache behavior. Implemented as a
ChunkHashLoggingSubscriber on the EventBus — no extra queue or
worker thread needed. Includes configurable log rotation, chunk
metadata (chunk_size, seq_len, dtypes, shapes), and CLI args.

Signed-off-by: Ryan <crclq2018@gmail.com>
Signed-off-by: rigginschen <rigginschen@tencent.com>

* refactor: rename ChunkHashLogger to LookupHashLogger

Rename the chunk hash logging subscriber to lookup hash logger to better
reflect that it logs hashes observed during lookup operations.

- chunk_hash.py → lookup_hash.py
- ChunkHashLogConfig → LookupHashLogConfig
- ChunkHashLoggingSubscriber → LookupHashLoggingSubscriber
- --chunk-hash-log-* CLI args → --lookup-hash-log-*
- lookup_hashes_*.jsonl file name pattern
- Update docs and tests accordingly

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: rigginschen <rigginschen@tencent.com>

* Use tell to get the accurate file size

Signed-off-by: rigginschen <rigginschen@tencent.com>

---------

Signed-off-by: rigginschen <rigginschen@tencent.com>
Signed-off-by: Ryan <crclq2018@gmail.com>
Co-authored-by: rigginschen <rigginschen@tencent.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant