Skip to content

Implement the internal last cache API #25093

@hiltontj

Description

@hiltontj

The last cache can be stored as a map of database/table/cache (there can be multiple named caches per table).

Each cache has a set of key columns (this is described well in this comment: #25109 (review)), and itself is a nested map, with one level for each key column in the hash. The nesting is done to create a hierarchy from the key columns if the cache.

Writes to the last cache will be sent after they have been flushed to the WAL, i.e., here.

The last cache needs to have an age out, or time-to-live (TTL) so that cache entries beyond a given age (default to 4 hours) are evicted, and the cache isn't bloated with old data.

The last cache implementation needs to also implement Datafusion's TableProvider trait, but that work may make more sense along with #25095.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions