Skip to content

[Topic-1] design: maxmemory ↔ compression interaction (R2.8.4) + R2.5.7 stickiness paragraph#36

Merged
ikolomi merged 1 commit into
unstablefrom
ikolomi/topic-1-maxmemory-docs
Jun 18, 2026
Merged

[Topic-1] design: maxmemory ↔ compression interaction (R2.8.4) + R2.5.7 stickiness paragraph#36
ikolomi merged 1 commit into
unstablefrom
ikolomi/topic-1-maxmemory-docs

Conversation

@ikolomi

@ikolomi ikolomi commented Jun 18, 2026

Copy link
Copy Markdown
Owner

Summary

Docs-only PR — adds R2.8.4 (maxmemory ↔ compression interaction prose) + a stickiness paragraph in R2.5.7. No code change. Lands as Topic-1 from implementation/plan.md, scheduled to follow Topic-2 PR-B so the prose could revisit whether anything new from PR-B's stress test justifies an additional metric.

What changes

design/detailed-design.md

R2.8.4 — Savings are workload-dependent; existing observability is sufficient. New requirement under §2.8 Memory accounting. Three sub-points:

  1. The same maxmemory yields a wide range of effective dataset capacity:
    • Compressible content → capacity grows by 1/ratio.
    • Incompressible content → unchanged minus per-value header (~16 B/key) and fixed pool overhead.
  2. Eviction interaction. Per R2.8.1 the sampler/MEMORY USAGE see compressed footprints. Eviction triggers at maxmemory measured in compressed bytes; victim selection by LRU/LFU is unchanged (per-key compression-ratio asymmetry doesn't bias which key is picked); evicted_keys (existing) is the canonical signal.
  3. Decompress-mode drain memory growth. Flipping compression-master-switch decompression progressively reverts values to RAW (R2.5.7's permanent-decompress mode + sweeper). used_memory grows toward the no-compression baseline. Operators with active maxmemory should ensure headroom before flipping; otherwise the drain may trigger eviction.

R2.5.7 — stickiness paragraph. Added inline (cross-referenced from R2.5.6). A value permanently decompressed — by drain-mode or by a write-path mutation through compressionPermanentlyDecompress — stays RAW until one of:

  • The sweeper visits the key on a subsequent tick under master=compression and the eligibility predicate (R2.2) admits it. The compression-min-idle-seconds gate means a freshly-mutated value waits one idle window before re-compressing — by design, to avoid churning the worker pool on hot keys.
  • The key is evicted, expired, or deleted — the stickiness question becomes moot.

No immediate auto-recompress path on permanent-decompress: re-compressing on every mutation would either churn the worker pool or compete for main-thread CPU; the sweep's pacing (compression-sweep-max-cpu-pct) bounds the re-compress overhead globally.

implementation/plan.md

Decision-record: no new metric introduced for v1

A "compression headroom" metric (or some equivalent of "effective uncompressed capacity") was considered after Topic-2 PR-B's stress test landed and rejected. Reasoning:

  • The existing INFO surface already lets operators reason about workload-dependent capacity:
    • used_memory (the standard one)
    • compression_ratio and compression_live_ratio_10m
    • compression_net_saved_bytes
    • compression_compressed_objects
    • compression_total_uncompressed_bytes / compression_total_compressed_bytes
    • evicted_keys (existing)
    • planned compression_transient_view_capped_total (S4.1, R2.5.7) for cap-fallback observability
  • PR-B's stress test surfaced no observation that would justify a new metric.

Revisit after S5.x benchmark scenarios run if a specific operator-facing gap surfaces.

Verification

Docs-only — no code change. git -P diff --stat shows 2 files, +17 / -2 lines (15 new lines in detailed-design.md, 2 plan.md edits + 2 lines).

CI's lint/clang-format/etc. should be unaffected.

….7 stickiness paragraph

R2.8.4 — savings are workload-dependent; existing observability is
sufficient. Documents three interactions Topic-2 PR-B (the integration
stress test) made it natural to revisit:

- The same maxmemory setting yields different effective capacity
  depending on whether the workload is compressible (capacity grows by
  1/ratio) or not (unchanged minus ~16 B/key header + fixed overhead).
- Eviction interaction: per R2.8.1 the sampler sees compressed
  footprints, so eviction triggers at maxmemory measured in compressed
  bytes; victim selection by LRU/LFU is unchanged (no bias from
  per-key ratio asymmetry); evicted_keys is the canonical signal.
- Decompress-mode drain causes used_memory to grow toward the
  no-compression baseline as the sweeper + permanent-decompress
  reclaim compressed frames; operators with active maxmemory should
  ensure headroom before flipping.

Decision-record: no new compression-specific metric introduced for v1.
PR-B's stress test surfaced no observation that would justify one.
The existing fields plus the planned compression_transient_view_capped_total
(S4.1) cover the operator-facing diagnostic surface. Revisit after
S5.x benchmark scenarios.

R2.5.7 — added stickiness paragraph cross-referenced from R2.5.6.
Spells out that a value permanently decompressed (drain-mode or
write-path mutation) stays RAW until either the sweeper visits it
under master=compression (gated by compression-min-idle-seconds, by
design to avoid worker-pool churn) or until eviction/expire/DEL
makes the question moot. No immediate auto-recompress path; the
sweeper pacing bounds the re-compress overhead globally.

plan.md updated:
- Topic-2 PR-B marked complete (merged at unstable@7013e370b via #35).
- Topic-1 marked complete (lands via this same PR).

Docs only — no code change. Build is unchanged.
@ikolomi ikolomi merged commit 0cea04d into unstable Jun 18, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant