Add CacheSQLite#13131
Conversation
Persist is intended for larger objects, which works best for things like cached tiles with some metadata. For many small to medium entries that you nevertheless don't necessarily want to keep around in memory, that seems a bit inefficient. SQLite then, is effectively a way of minimizing the amount of writes in this scenario. Relates to koreader#13061. Depends on koreader#13122.
|
Sounds good! Random query: would an in-memory db variant make sense in some of those use-cases? |
| UPDATE cache SET last_access = ? WHERE key = ? | ||
| RETURNING value; | ||
| ]]) | ||
| local row = stmt:reset():bind(os.time(), key):step() |
There was a problem hiding this comment.
(Comment valid across all the os.time calls)
time.now instead?
Or even UIManager:getTime() if a coarser, cached value from our last UI frame is sufficient?
There was a problem hiding this comment.
I like that os.time() is human-legible when looking at the table, fwiw. UIManager:getTime() probably makes the most sense as an alternative, or in any case I don't see what higher granularity could be desired for.
I toyed with something like |
|
Come to think of it, it'd probably be better to add an |
| self.db:exec[[ | ||
| CREATE TABLE IF NOT EXISTS cache ( | ||
| key TEXT PRIMARY KEY, | ||
| value BLOB, |
There was a problem hiding this comment.
I departed from Cache because it mostly met my needs. It's just that the writing of the cache to disk would've resulted in many excess writes. But while writing it I therefore didn't consider that this part could be more dynamic to take better advantage of SQLite. I don't think it matters in the near future though.
Edit: by which I mean maybe for some use case you don't need any blobs, maybe you just need a few text columns instead.
There was a problem hiding this comment.
https://www.sqlite.org/datatype3.html
SQLite uses a more general dynamic type system. In SQLite, the datatype of a value is associated with the value itself, not with its container.
So, it's my understanding the column type you set in a CREATE TABLE is just documentation, and may be value CRAP works :)
There was a problem hiding this comment.
I'm not referring to the types, or at least not exactly, just the number of columns and their names. This type is effectively "there's only 1 column so you can't sanely use SQL on it".
There was a problem hiding this comment.
I then still don't understand what you are referring to :)
What would be the alternative you end up not using/needing?
There was a problem hiding this comment.
I didn't think of it until just last night when I wrote that, but the cache in question could define its own custom columns. My entire thinking was based around passing in serialized Lua objects.
If your object is merely {title="this", author="that"}, maybe you'd be better served with two regular columns. I'm not entirely sure how useful such a thing would be, but there are some potential secondary use cases based on what's in the cache that'd be more efficient written in SQL than by doing a lot of Lua looping asking for rows one at a time.
In that case the cache insertion and associated auto-deletion logic would still be the reason to use the cache, but for certain more advanced interactions you could do something like cache:getDBConn():prepare(etc) to run custom queries on the data.
Persist is intended for larger objects, which works best for things like cached tiles with some metadata. For many small to medium entries that you nevertheless don't necessarily want to keep around in memory, that seems a bit inefficient. SQLite then, is effectively a way of minimizing the amount of writes in this scenario. Relates to koreader#13061. Depends on koreader#13122.
Persist is intended for larger objects, which works best for things like cached tiles with some metadata.
For many small to medium entries that you nevertheless don't necessarily want to keep around in memory, that seems a bit inefficient.
SQLite then, is effectively a way of minimizing the amount of writes in this scenario.
Relates to #13061. Depends on #13122.
This change is