fix(core): flush in-memory SQL databases to file on actor shutdown#40
Merged
fix(core): flush in-memory SQL databases to file on actor shutdown#40
Conversation
SQL database actors start with an in-memory SQLite instance and only promote to file storage when data exceeds the memory_threshold (10 MiB). Small databases never hit this threshold, so when the actor shuts down after the 5-minute idle timeout, all data is silently lost. This adds a flush step just before the actor exits: if the database is still in-memory and contains any data, it is promoted to file via the existing SQLite backup API. On the next request, the actor sees the file and opens from disk — data survives across idle cycles. KV is unaffected because it writes to the server's main persistent database immediately. This only impacts the per-space SQLite service. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
SQL database actors start with an in-memory SQLite instance and only promote to file storage when data exceeds the
memory_threshold(default: 10 MiB). For small databases — like a user with a handful of rows — data never crosses this threshold.When the actor hits the 5-minute idle timeout, it shuts down and the in-memory data is silently lost. The next request spawns a fresh actor with an empty database. Users experience this as intermittent data loss: "my SQL data disappeared."
KV storage is unaffected because it writes to the server's main persistent database (sea-orm) immediately. This only impacts the per-space SQLite service.
Root Cause
In
tinycloud-core/src/sql/database.rs, the actor loop exits on idle timeout (IDLE_TIMEOUT = 300s) and proceeds directly to cleanup without persisting in-memory state:The
promote_to_filepath only runs during the loop, gated bysize > memory_threshold.Fix
After the actor loop breaks, check if the database is still in-memory. If it contains any data, flush it to disk via the existing
promote_to_filefunction (SQLite backup API) before the actor exits.On the next request,
spawn_actorsees the file on disk and opens from file — data survives across idle cycles.What this doesn't change
memory_thresholdpromotion-during-writes path is untouched — large databases still promote eagerlyTest plan
cargo checkpasses ontinycloud-corecargo test sqlpasses"Flushed in-memory database to file on shutdown"after idle timeout🤖 Generated with Claude Code