Skip to content

Conversation

@Ayyanaruto
Copy link
Contributor

@Ayyanaruto Ayyanaruto commented Oct 21, 2025

Based on discussion #8315 Implement asynchronous initialization for the autocomplete cache and refine eviction logic to optimize performance and memory usage. This update ensures a more efficient handling of cached entries with SQLite persistence.


Summary by cubic

Initialize the autocomplete cache asynchronously and move lookups to an in-memory LRU with periodic SQLite persistence. This improves startup, speeds up cache hits, and tightens eviction for better memory usage.

  • New Features
    • Async singleton cache initialization that loads existing entries and starts a 30s flush timer.
    • In-memory longest-prefix matching with validation; timestamps updated on access.
    • LRU eviction in memory when capacity is exceeded, marking evicted keys for deletion.
    • Periodic flush persists dirty entries in a single transaction (upsert/delete) under a mutex, with graceful error handling.
    • CompletionProvider now awaits the cache on demand and disposes it cleanly via close().
    • Added comprehensive tests for cache operations, eviction, persistence, timer behavior, and resource cleanup.

@Ayyanaruto Ayyanaruto requested a review from a team as a code owner October 21, 2025 10:25
@Ayyanaruto Ayyanaruto requested review from sestinj and removed request for a team October 21, 2025 10:25
@dosubot dosubot bot added the size:XL This PR changes 500-999 lines, ignoring generated files. label Oct 21, 2025
@Ayyanaruto Ayyanaruto changed the title Enhance autocomplete cache with async initialization and improved eviction logic feat: Enhance autocomplete cache with async initialization and improved eviction logic Oct 21, 2025
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 issues found across 3 files

Prompt for AI agents (all 3 issues)

Understand the root cause of the following 3 issues and fix them.


<file name="core/autocomplete/util/AutocompleteLruCache.ts">

<violation number="1" location="core/autocomplete/util/AutocompleteLruCache.ts:218">
After close(), we need to clear the singleton so future get() calls reinitialize a fresh cache; otherwise callers receive an instance whose DB connection is already closed and whose flush timer never restarts.</violation>
</file>

<file name="core/autocomplete/util/AutocompleteLruCache.test.ts">

<violation number="1" location="core/autocomplete/util/AutocompleteLruCache.test.ts:4">
Mocking `async-mutex` removes the real Mutex implementation, so `cache.put` fails when it tries to call `this.mutex.acquire()`.</violation>

<violation number="2" location="core/autocomplete/util/AutocompleteLruCache.test.ts:638">
`jest.doMock(&quot;sqlite&quot;, …)` runs after AutocompleteLruCache imported `open`, leaving `open` as the default auto-mock and causing `AutocompleteLruCache.get()` to crash with an undefined database handle.</violation>
</file>

React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Nov 12, 2025
@github-project-automation github-project-automation bot moved this from Todo to In Progress in Issues and PRs Nov 12, 2025
@RomneyDa
Copy link
Collaborator

@Ayyanaruto appreciate this contribution!

@Ayyanaruto
Copy link
Contributor Author

Pleasure is all mine

@RomneyDa RomneyDa merged commit 6856d9b into continuedev:main Nov 19, 2025
55 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Issues and PRs Nov 19, 2025
@github-actions github-actions bot locked and limited conversation to collaborators Nov 19, 2025
@sestinj
Copy link
Contributor

sestinj commented Nov 19, 2025

🎉 This PR is included in version 1.5.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@sestinj
Copy link
Contributor

sestinj commented Nov 19, 2025

🎉 This PR is included in version 1.31.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@sestinj
Copy link
Contributor

sestinj commented Nov 20, 2025

🎉 This PR is included in version 1.35.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@sestinj
Copy link
Contributor

sestinj commented Nov 20, 2025

🎉 This PR is included in version 1.6.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions github-actions bot deleted the f/optimize-autocompletelru-cache branch December 22, 2025 06:17
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

lgtm This PR has been approved by a maintainer released size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants