Addresses 5 of 6 review comments on the QSBR design. Comment #6
(`compressionJob.key` extra-lookup concern) is explicitly deferred to
a follow-up PR per reviewer guidance.

Comment #1 (line 428) and #5 (line 544) — drop language-comparison
framing:
  Removed all references to Rust / `Arc<T>` / "memory-safe languages"
  / `shared_ptr` from §4.4 intro, the "Why QSBR" bullet list, and the
  §4.6 "Why the worker loads the active dict itself" paragraph. The
  rationale now stands on its own technical merit (decoupling the
  registry from worker hot paths; minimal worker contract;
  safe-directional failure modes) rather than via comparison to
  another language's type system. C with explicit protocols is the
  right tool for this problem; the comparison added rhetorical
  weight without adding signal.

Comment #2 (line 326) — duplication with R2.11.4:
  §3.3 Separation invariants restated the worker contract that
  R2.11.4 already specifies authoritatively. Slimmed the §3.3 bullet
  to a one-liner that points at §2.11 R2.11.4 and §4.4. Eliminates
  drift risk between the two places.

Comment #3 (line 439) — bound the retiring list, block on cap:
  Added new step 7 to the QSBR section explaining the cap interaction
  with R2.3.3. The retiring list is a subset of `dicts[]`, capped at
  `compression-dict-max-versions`. When grace-barrier draining cannot
  keep up (worker starvation, persistent `frame_refs > 0`), the cap
  is reached and BOTH training AND promotion are refused per R2.3.3:
  `LL_WARNING` log entry, `compression_dict_cap_reached` set in INFO,
  operator intervention required (raise cap or run COMPRESSION SWEEP).

Comment #4 (line 449) — grace-barrier wake-up via cond_broadcast:
  The original step 6 proposed enqueueing barrier jobs into the SPMC
  inbox to force idle workers to advance generations. This doesn't
  actually work: under work-stealing semantics a single fast worker
  can drain all barrier jobs while siblings stay asleep on the cond
  var. Rewrote step 6 to use a wake-all primitive built on
  `pthread_cond_broadcast`, and added a "Wake-all primitive"
  paragraph to §4.6 that describes extending `mutexqueue.h` with two
  new APIs: a broadcast wake-all (for QSBR grace barriers, config
  changes, etc.) and a shutdown-signal variant (for pool teardown).
  Step 6 cross-references §4.6 for the mechanism.

Comment #6 (line 513) — DEFERRED:
  Reviewer flagged that `compressionJob.key` (a `robj *` carried in
  the job) implies the main thread does an additional lookup at
  install time, doubling the per-write lookup cost. The reviewer
  explicitly tagged this as "follow up PR" — addressing it would
  require a redesign of the install-side data flow and is out of
  scope for the QSBR design change. Tracked as an open item; will
  be addressed before code lands for the install path (S2.7 in
  the implementation plan).

Two reviewer threads addressed:

Thread #1 (T-3369017721) — production code carrying test concerns

  The drain handler had a `if (job->value == NULL)` branch that only
  existed to handle test-only jobs from
  testOnlyCompressionWorkersEnqueueRaw. Reviewer correctly pointed out
  that production code shouldn't carry test-only branches.

  Fix: replaced with serverAssert(job->value != NULL) at the top of
  the per-job loop. Production drain assumes every job has a real
  pinned robj; tests must extract their value=NULL jobs via
  testOnlyCompressionWorkersDrainOutbox before this drain runs.

  Side effect: removed the conditional `if (job->value != NULL)`
  guards around decrRefCount and the install branch — the top-of-loop
  assert means every code path can assume value is non-NULL.

Thread #2 (T-3356207626) — design doc out of sync with implementation

  Design §4.6 still described the original version-counter approach
  for staleness detection (`uint64_t version` field on compressionJob,
  "if version counter moved, discard"). The implementation has used
  pointer equality + the incrRefCount-pin since S2.4 PR #13.

  Fix: updated §4.6 to:
    - compressionJob struct: drop `version`, drop `robj *key`, add
      `robj *value` (pinned via incrRefCount), and `sds src` and
      `int dbid` separately, matching the actual struct.
    - Concurrency notes: replaced the "version counter moved" bullet
      with the pointer-equality + ABA-safety reasoning, naming the
      incrRefCount-reserves-the-address invariant as the protection
      mechanism (same property explained in PR #18 review).

Verified locally:
  - make -j2 -C src              → clean
  - ./runtest --single unit/type/compression  → 10/10 pass

* [S2.7] Compression write-path hook

Wires compressionEnqueueCandidate into dbAddInternal and dbSetValue,
and replaces the TODO(S2.7) placeholder in the drain handler with a
real install path. With this change, writes to eligible STRING values
get queued for background compression and the result is installed back
into the kvstore as an OBJ_ENCODING_COMPRESSED robj.

The decoder (S2.6) is shipped but not yet wired into read paths (S2.8),
so as long as compression-enabled stays no (default), behavior is
unchanged. Once an operator turns the switch on, written values get
compressed, but reads return the compressed bytes until S2.8 lands.
Existing transparency tests verify no regression in the default-off
configuration.

Producer side (compression.c, db.c)

  Two seams in db.c — end of dbAddInternal and end of dbSetValue —
  call compressionEnqueueCandidate(key, value, db->id). The candidate
  function applies four guards:
    1. Master switch (compression_enabled, via compressionIsEligible).
    2. R2.2 eligibility (type/encoding/size/hot-key — also via predicate).
    3. R2.1.5 active-dict check — saves an allocator round-trip when
       compression-enabled=yes but training hasn't completed.
    4. incrRefCount(value) — pins the bytes for the worker AND
       reserves the robj address for the drain handler's pointer-
       equality stale check (ABA-safe per R2.4.4 + the lifetime
       discussion in PR #18).

  If the worker pool refuses (not started; future S2.11 inbox full),
  the pin is released immediately. RDB-load enqueue is deliberately
  skipped — TODO(S2.10): the sweep tick will rediscover RDB-loaded
  values without hammering the inbox during load.

API change: compressionWorkersEnqueue

  Old: compressionWorkersEnqueue(sds key, int dbid, uint64_t version, sds src)
  New: compressionWorkersEnqueue(robj *value, int dbid)

  The new form requires a pinned robj; the worker reads
  objectGetVal(value) once at enqueue (captured into job->src) and
  never touches the robj afterwards (R2.11.4 intact). The drain
  handler uses job->value for the kvstore lookup and the pointer-
  equality stale check.

  The version field is gone — pointer equality, made ABA-safe by the
  pin, is sufficient. R2.4.4 explains why: holding incrRefCount(value)
  prevents the allocator from reusing the address while the job is
  in flight.

Drain install (compression_workers.c)

  New compressionInstall() helper:
    1. void **slot = kvstoreHashtableFindRef(db->keys, didx, key_sds);
    2. If slot == NULL OR *slot != job->value: stale (overwrite, expire,
       or COW). Discard.
    3. Else: createCompressedObject(OBJ_STRING, job->dst, job->dst_len);
       dbReplaceValue installs.
    4. compressionRegistryIncRef(job->dict_id) on success.

  dbReplaceValue routes through dbSetValue(..., overwrite=0, ...),
  which does NOT call signalModifiedKey, moduleNotifyKeyUnlink, or
  signalDeletedKeyAsReady. Background compression is a storage-only
  change per R2.9.2 — no WATCH dirty_cas, no client-side-caching
  invalidations, no keyspace notifications.

  Pin released on every drain completion path (success, stale-discard,
  net-savings reject, ZSTD error, no-active-dict). Test-mode jobs
  (job->value == NULL) skip both install and decRef.

Test migration

  The 15 existing test-fixture call sites passed raw sds + dummy
  version. Migrated to a new testOnlyCompressionWorkersEnqueueRaw(src,
  dbid) that sets job->value = NULL. Tests extract jobs via
  testOnlyCompressionWorkersDrainOutbox before the production drain
  runs, so production-only paths (install, decRef) are never reached
  by the value=NULL sentinel.

  No new gtest cases for the install path itself — that requires a
  fully-initialized server.db / kvstore that the unit-test environment
  doesn't construct. End-to-end coverage will come from the Tcl
  transparency harness once S2.8 wires the read path.

TODO(S4.1) markers added at:
  - compressionInstall: compression_compressions_per_sec, EMA fold,
    compression_compressed_objects.
  - compressionEnqueueCandidate: compression_candidates_dropped_total
    when S2.11 lands (today the pool-not-started rejection is a
    config state, not back-pressure).

Verified locally:
  - make -j2 -C src              → clean (BUILD_ZSTD=yes default).
  - make -j2 -C src BUILD_ZSTD=no → clean.
  - ./runtest --single unit/type/compression → 10/10 pass.

  gtest unit tests not runnable locally; CI validates.

Diff stat:

  .../implementation/plan.md                |   4 +-
  src/compression.c                         |  35 +++-
  src/compression.h                         |  27 ++-
  src/compression_workers.c                 | 185 +++++++++++++++------
  src/compression_workers.h                 |  56 +++----
  src/db.c                                  |  14 ++
  src/unit/test_compression_workers.cpp     |  31 ++--
  7 files changed, 244 insertions(+), 108 deletions(-)

* [S2.7] PR #19 review: assert + design-doc alignment

Two reviewer threads addressed:

Thread #1 (T-3369017721) — production code carrying test concerns

  The drain handler had a `if (job->value == NULL)` branch that only
  existed to handle test-only jobs from
  testOnlyCompressionWorkersEnqueueRaw. Reviewer correctly pointed out
  that production code shouldn't carry test-only branches.

  Fix: replaced with serverAssert(job->value != NULL) at the top of
  the per-job loop. Production drain assumes every job has a real
  pinned robj; tests must extract their value=NULL jobs via
  testOnlyCompressionWorkersDrainOutbox before this drain runs.

  Side effect: removed the conditional `if (job->value != NULL)`
  guards around decrRefCount and the install branch — the top-of-loop
  assert means every code path can assume value is non-NULL.

Thread #2 (T-3356207626) — design doc out of sync with implementation

  Design §4.6 still described the original version-counter approach
  for staleness detection (`uint64_t version` field on compressionJob,
  "if version counter moved, discard"). The implementation has used
  pointer equality + the incrRefCount-pin since S2.4 PR #13.

  Fix: updated §4.6 to:
    - compressionJob struct: drop `version`, drop `robj *key`, add
      `robj *value` (pinned via incrRefCount), and `sds src` and
      `int dbid` separately, matching the actual struct.
    - Concurrency notes: replaced the "version counter moved" bullet
      with the pointer-equality + ABA-safety reasoning, naming the
      incrRefCount-reserves-the-address invariant as the protection
      mechanism (same property explained in PR #18 review).

Verified locally:
  - make -j2 -C src              → clean
  - ./runtest --single unit/type/compression  → 10/10 pass

* [S2.7] Fix CI: remove erroneous & on server.db indexing

build-32bit (and the 30+ downstream cells, all CI cells use -Werror):

  compression_workers.c:531:20: error: initialization of 'serverDb *'
    from incompatible pointer type 'serverDb **'
    [-Werror=incompatible-pointer-types]

`server.db` is `serverDb **` (array of pointers, one per DB). So
`server.db[i]` is already `serverDb *` — the address-of operator was
redundant and produced `serverDb **`.

Fix: drop the `&`. Matches the pattern used everywhere else in the
codebase (db.c, server.c, etc.).

Local make didn't catch this — the default SERVER_CFLAGS doesn't
include -Werror. CI does. Built locally with `make SERVER_CFLAGS=-Werror`
to confirm clean.

* [S2.7] Fix CI: tests must use testOnly drain for value=NULL jobs

5 gtest cases failed on build-32bit (and would on every test cell)
with the new production-drain serverAssert(job->value != NULL):

  ASSERTION FAILED: compression_workers.c:591 'job->value != NULL'

  in: SingleJobRoundTrip, BurstOf256JobsOneWorker,
      BurstOf1024JobsFourWorkers, ResizeAcrossEnqueuedJobs,
      NetSavingsGuardRejectsIncompressible

Root cause: the previous commit's reviewer-driven hardening (PR #19
review thread #1) made the production drain assert that every job
has a non-NULL pinned robj. The premise was "tests use the testOnly
drain to extract jobs before the production drain runs". That premise
was wrong — many tests ALSO call compressionWorkersDrainOutbox
directly to consume-and-dispose test-mode jobs (the drainUntil helper
is the most-used path).

Fix: add testOnlyCompressionWorkersDrainAndDispose(budget) — pulls
jobs via the existing testOnlyCompressionWorkersDrainOutbox, frees
them via testOnlyCompressionWorkersFreeJob, returns count. Migrate
the test fixture's drainUntil helper and all 8 direct
compressionWorkersDrainOutbox call sites in the test file to the
new helper.

Production drain stays clean — no test concerns. Reviewer thread #1
intent preserved.

Verified locally:
  - make -j2 -C src SERVER_CFLAGS=-Werror   → clean
  - ./runtest --single unit/type/compression → 10/10 pass

… fix

Adds tests/integration/compression.tcl — first end-to-end exercise of
the merged S2.x stack against a real workload. Builds on top of PR-A
(#33)'s runtime dict-generation infrastructure: each test imports a
freshly-trained dict via tests/support/compression-helpers.tcl, then
exercises one specific behaviour of the hot path.

Five test cases:

  1. Write-path round trip.
     master=compression + sweeper=disabled. SET a compressible value,
     poll until OBJECT ENCODING reports "compressed", verify GET
     round-trips the original bytes through the read-path transient
     view (R2.5.7).

  2. Sweeper compresses pre-existing keys.
     master=off + populate 100 RAW keys, then flip to
     master=compression + sweeper=enabled. Poll until at least 80% of
     the population is compressed via the sweeper's automatic pass.
     Spot-check 5 sample keys round-trip cleanly.

  3. Decompression drain.
     Continuing from #2's compressed state, flip to
     master=decompression + sweeper=enabled. Poll until 0 keys remain
     compressed. Verify reads still work and encoding is "raw".

  4. COMPRESSION SWEEP FORCE end-to-end.
     master=compression + sweeper=disabled (so the only trigger is
     manual). Populate uncompressed, then run COMPRESSION SWEEP FORCE.
     Verify keys get compressed by a single forced pass.

  5. Mixed workload preserves data integrity under live sweeper.
     master=compression + sweeper=enabled + 50% pacing. Populate 200
     keys, run 500 random ops (GET/SET/APPEND/EXPIRE/OBJECT ENCODING/
     DEL) using the deterministic LCG from PR-A. Track expected value
     per key in a Tcl dict; final pass asserts every surviving key
     round-trips. Asserts compression_errors_total == 0.

All 5 tests skip cleanly under BUILD_ZSTD=no (helper binary not built).

Prerequisite fix: src/object.c strEncoding()
---------------------------------------------

`OBJECT ENCODING <key>` was returning "unknown" for compressed values
because strEncoding() didn't have a case for OBJ_ENCODING_COMPRESSED.
Design R2.7.1 requires it returns "compressed". This is a one-line
fix that should have landed with createCompressedObject in S2.5; it
slipped through. Adding the missing case here as a prerequisite,
since every test in this file polls OBJECT ENCODING.

Polling strategy
----------------

`compression_compressed_objects` in INFO is currently hardcoded to 0
(noted in the renderer comment as "stays 0 until later S2 PRs land
their counters" — depends on encode-path counters tracked separately
in S4.x). Tests therefore can't poll the counter; they poll OBJECT
ENCODING per-key instead via three helpers in compression.tcl:

  - wait_for_encoding key expected_encoding
  - wait_for_at_least_n_keys_with_encoding keys target encoding
  - wait_for_at_most_n_keys_with_encoding keys target encoding

The helpers use the existing wait_for_condition primitive with a
50ms × 200-tries default budget — gives the server's event loop
time to drain the worker outbox between polls. When the counter
gets wired in a future S2 PR, these helpers can be simplified or
augmented but the API stays the same.

Verification
------------

  - 392 gtests pass (no change vs PR-A).
  - 33 Tcl tests pass: 28 unit/type/compression (PR-A) + 5
    integration/compression (this PR).
  - BUILD_ZSTD=yes: all tests run.
  - BUILD_ZSTD=no: integration tests skip cleanly with a single
    skip stub; unit/type/compression skips its helper-dependent
    tests as before.
  - Both flavors compile clean with -Werror.

Out of scope for this PR
------------------------

  - Drift / retraining tests (need S1.x's training engine landed
    end-to-end, plus design discussion on what "drift detected"
    looks like at the test level).
  - Counter wiring (compression_compressed_objects et al) — separate
    S4.x ticket.
  - COW invariant test (S2.13 / S6.1).

… fix

Adds tests/integration/compression.tcl — first end-to-end exercise of
the merged S2.x stack against a real workload. Builds on top of PR-A
(#33)'s runtime dict-generation infrastructure: each test imports a
freshly-trained dict via tests/support/compression-helpers.tcl, then
exercises one specific behaviour of the hot path.

Six test cases:

  1. Write-path round trip.
     master=compression + sweeper=disabled. SET a compressible value,
     poll until OBJECT ENCODING reports "compressed", verify GET
     round-trips the original bytes through the read-path transient
     view (R2.5.7).

  2. Sweeper compresses pre-existing keys.
     master=off + populate 100 RAW keys, then flip to
     master=compression + sweeper=enabled. Wait for ALL keys
     compressed; verify EVERY value round-trips (no spot-checks).

  3. Decompression drain.
     Continuing from #2's compressed state, flip to
     master=decompression + sweeper=enabled. Wait for ALL keys
     drained back to RAW; verify EVERY value round-trips.

  4. COMPRESSION SWEEP FORCE end-to-end.
     master=compression + sweeper=disabled (manual-only). Populate
     uncompressed, then run COMPRESSION SWEEP FORCE. Wait for ALL
     keys to be compressed by a single forced pass; verify EVERY
     value round-trips.

  5. Mixed workload preserves data integrity under live sweeper.
     master=compression + sweeper=enabled + 50% pacing. Populate 200
     keys, run 500 random ops (GET/SET/APPEND/SETRANGE/DEL — see the
     `# Mixed ops` comment in the file for the rationale of each op,
     SETRANGE replaced EXPIRE in this revision since EXPIRE doesn't
     touch the value bytes and isn't relevant to compression
     correctness). Track per-key expected value as a Tcl dict; final
     pass asserts every surviving key round-trips.

  6. Ineligibility — values outside the size envelope and hot keys.
     New test added per review feedback. Exercises the eligibility
     predicate (R2.2): values below `compression-min-value-size`,
     values above `compression-max-value-size`, and freshly-written
     "hot" keys (`lru_idle_secs(obj) < compression-min-idle-seconds`)
     must NOT be compressed even with the sweeper running at maximum
     cadence. A control key with a relaxed idle threshold proves the
     sweeper is actually running, ruling out the false-positive of
     "everything was skipped because the sweeper crashed".

All 6 tests skip cleanly under BUILD_ZSTD=no (helper binary not built).

Prerequisite fix: src/object.c strEncoding()
---------------------------------------------

`OBJECT ENCODING <key>` was returning "unknown" for compressed values
because strEncoding() didn't have a case for OBJ_ENCODING_COMPRESSED.
Design R2.7.1 requires it returns "compressed". One-line fix that
should have landed with createCompressedObject in S2.5; slipped
through. Adding the missing case here as a prerequisite, since every
test in this file polls OBJECT ENCODING.

Polling strategy
----------------

`compression_compressed_objects` in INFO is currently hardcoded to 0
(noted in the renderer comment as "stays 0 until later S2 PRs land
their counters" — depends on encode-path counters tracked separately
in S4.x). Tests therefore can't poll the counter; they poll OBJECT
ENCODING per-key instead via three helpers in compression.tcl. Each
helper carries an explicit TODO(S4.x) annotation per the project's
new "TODO-mark suboptimal/superseded code" rule (review feedback).
Same TODO marker on assert_no_compression_errors, since
compression_errors_total is also currently hardcoded to 0.

Verification
------------

  - 392 gtests pass.
  - 34 Tcl tests pass: 28 unit/type/compression (PR-A) + 6
    integration/compression (this PR).
  - BUILD_ZSTD=yes: all tests run.
  - BUILD_ZSTD=no: integration tests skip cleanly with a single
    skip stub.
  - Both flavors compile clean with -Werror.

… fix

Adds tests/integration/compression.tcl — first end-to-end exercise of
the merged S2.x stack against a real workload. Builds on top of PR-A
(#33)'s runtime dict-generation infrastructure: each test imports a
freshly-trained dict via tests/support/compression-helpers.tcl, then
exercises one specific behaviour of the hot path.

Six test cases:

  1. Write-path round trip.
     master=compression + sweeper=disabled. SET a compressible value,
     poll until OBJECT ENCODING reports "compressed", verify GET
     round-trips the original bytes through the read-path transient
     view (R2.5.7).

  2. Sweeper compresses pre-existing keys.
     master=off + populate 100 RAW keys, then flip to
     master=compression + sweeper=enabled. Wait for ALL keys
     compressed; verify EVERY value round-trips (no spot-checks).

  3. Decompression drain.
     Continuing from #2's compressed state, flip to
     master=decompression + sweeper=enabled. Wait for ALL keys
     drained back to RAW; verify EVERY value round-trips.

  4. COMPRESSION SWEEP FORCE end-to-end.
     master=compression + sweeper=disabled (manual-only). Populate
     uncompressed, then run COMPRESSION SWEEP FORCE. Wait for ALL
     keys to be compressed by a single forced pass; verify EVERY
     value round-trips.

  5. Mixed workload preserves data integrity under live sweeper.
     master=compression + sweeper=enabled + 50% pacing. Populate 200
     keys, run 500 random ops (GET/SET/APPEND/SETRANGE/DEL — see the
     `# Mixed ops` comment in the file for the rationale of each op,
     SETRANGE replaced EXPIRE in this revision since EXPIRE doesn't
     touch the value bytes and isn't relevant to compression
     correctness). Track per-key expected value as a Tcl dict; final
     pass asserts every surviving key round-trips.

  6. Ineligibility — values outside the size envelope and hot keys.
     New test added per review feedback. Exercises the eligibility
     predicate (R2.2): values below `compression-min-value-size`,
     values above `compression-max-value-size`, and freshly-written
     "hot" keys (`lru_idle_secs(obj) < compression-min-idle-seconds`)
     must NOT be compressed even with the sweeper running at maximum
     cadence. A control key with a relaxed idle threshold proves the
     sweeper is actually running, ruling out the false-positive of
     "everything was skipped because the sweeper crashed".

All 6 tests skip cleanly under BUILD_ZSTD=no (helper binary not built).

Prerequisite fix: src/object.c strEncoding()
---------------------------------------------

`OBJECT ENCODING <key>` was returning "unknown" for compressed values
because strEncoding() didn't have a case for OBJ_ENCODING_COMPRESSED.
Design R2.7.1 requires it returns "compressed". One-line fix that
should have landed with createCompressedObject in S2.5; slipped
through. Adding the missing case here as a prerequisite, since every
test in this file polls OBJECT ENCODING.

Polling strategy
----------------

`compression_compressed_objects` in INFO is currently hardcoded to 0
(noted in the renderer comment as "stays 0 until later S2 PRs land
their counters" — depends on encode-path counters tracked separately
in S4.x). Tests therefore can't poll the counter; they poll OBJECT
ENCODING per-key instead via three helpers in compression.tcl. Each
helper carries an explicit TODO(S4.x) annotation per the project's
new "TODO-mark suboptimal/superseded code" rule (review feedback).
Same TODO marker on assert_no_compression_errors, since
compression_errors_total is also currently hardcoded to 0.

Verification
------------

  - 392 gtests pass.
  - 34 Tcl tests pass: 28 unit/type/compression (PR-A) + 6
    integration/compression (this PR).
  - BUILD_ZSTD=yes: all tests run.
  - BUILD_ZSTD=no: integration tests skip cleanly with a single
    skip stub.
  - Both flavors compile clean with -Werror.

…ite fixes

Adds tests/integration/compression.tcl — first end-to-end exercise of
the merged S2.x stack against a real workload. Builds on top of PR-A
(#33)'s runtime dict-generation infrastructure: each test imports a
freshly-trained dict via tests/support/compression-helpers.tcl, then
exercises one specific behaviour of the hot path.

Six test cases:

  1. Write-path round trip.
     master=compression + sweeper=disabled. SET a compressible value,
     poll until OBJECT ENCODING reports "compressed", verify GET
     round-trips the original bytes through the read-path transient
     view (R2.5.7).

  2. Sweeper compresses pre-existing keys.
     master=off + populate 100 RAW keys, then flip to
     master=compression + sweeper=enabled. Wait for ALL keys
     compressed; verify EVERY value round-trips (no spot-checks).

  3. Decompression drain.
     Continuing from #2's compressed state, flip to
     master=decompression + sweeper=enabled. Wait for ALL keys
     drained back to RAW; verify EVERY value round-trips.

  4. COMPRESSION SWEEP FORCE end-to-end.
     master=compression + sweeper=disabled (manual-only). Populate
     uncompressed, then run COMPRESSION SWEEP FORCE. Wait for ALL
     keys to be compressed by a single forced pass; verify EVERY
     value round-trips.

  5. Mixed workload preserves data integrity under live sweeper.
     master=compression + sweeper=enabled + 50% pacing. Populate 200
     keys, run 500 random ops (GET/SET/APPEND/SETRANGE/DEL).

  6. Ineligibility — values outside the size envelope and hot keys.
     Verifies the eligibility predicate (R2.2): values below
     compression-min-value-size, values above compression-max-value-size,
     and freshly-written hot keys must NOT be compressed even with the
     sweeper running at maximum cadence.

Prerequisite fix 1: src/object.c strEncoding()
================================================

OBJECT ENCODING was returning "unknown" for compressed values
because strEncoding() didn't have a case for OBJ_ENCODING_COMPRESSED.
Design R2.7.1 requires it returns "compressed". One-line fix that
slipped through earlier S2 PRs.

Prerequisite fix 2: src/compression.c compressionEnqueueCandidate()
====================================================================

Use-after-free caught by AddressSanitizer on the first PR-B CI run.
The eligibility predicate accepts encoding==RAW; a value currently
in transient-view state (R2.5.7) reads as RAW because val_ptr is
the per-iteration temp uncompressed sds. compressionEnqueueCandidate
would then capture job->src = temp_sds, which restoreTransientEntry
frees at beforeSleep — leaving the worker thread's job->src dangling
into freed memory.

Fix: gate the enqueue on !transientViewActive(value). Skipping the
enqueue is functionally harmless — a value in transient view is
already compressed (the original frame is saved in the side-map and
will be restored at beforeSleep). One line added at the top of
compressionEnqueueCandidate, with an explanatory comment naming the
exact ASan trace it fixes.

Tests: 392 gtests + 34 Tcl tests pass (28 unit + 6 integration).
Both BUILD_ZSTD={yes,no} build clean with -Werror. Verified the
asan fix locally by rebuilding with -fsanitize=address and re-running
the integration suite — no use-after-free.

…ite fixes

Adds tests/integration/compression.tcl — first end-to-end exercise of
the merged S2.x stack against a real workload. Builds on top of PR-A
(#33)'s runtime dict-generation infrastructure: each test imports a
freshly-trained dict via tests/support/compression-helpers.tcl, then
exercises one specific behaviour of the hot path.

Six test cases:

  1. Write-path round trip.
     master=compression + sweeper=disabled. SET a compressible value,
     poll until OBJECT ENCODING reports "compressed", verify GET
     round-trips the original bytes through the read-path transient
     view (R2.5.7).

  2. Sweeper compresses pre-existing keys.
     master=off + populate 100 RAW keys, then flip to
     master=compression + sweeper=enabled. Wait for ALL keys
     compressed; verify EVERY value round-trips (no spot-checks).

  3. Decompression drain.
     Continuing from #2's compressed state, flip to
     master=decompression + sweeper=enabled. Wait for ALL keys
     drained back to RAW; verify EVERY value round-trips.

  4. COMPRESSION SWEEP FORCE end-to-end.
     master=compression + sweeper=disabled (manual-only). Populate
     uncompressed, then run COMPRESSION SWEEP FORCE. Wait for ALL
     keys to be compressed by a single forced pass; verify EVERY
     value round-trips.

  5. Mixed workload preserves data integrity under live sweeper.
     master=compression + sweeper=enabled + 50% pacing. Populate 200
     keys, run 500 random ops (GET/SET/APPEND/SETRANGE/DEL).

  6. Ineligibility — values outside the size envelope and hot keys.
     Verifies the eligibility predicate (R2.2): values below
     compression-min-value-size, values above compression-max-value-size,
     and freshly-written hot keys must NOT be compressed even with the
     sweeper running at maximum cadence.

Prerequisite fix 1: src/object.c strEncoding()
================================================

OBJECT ENCODING was returning "unknown" for compressed values
because strEncoding() didn't have a case for OBJ_ENCODING_COMPRESSED.
Design R2.7.1 requires it returns "compressed". One-line fix that
slipped through earlier S2 PRs.

Prerequisite fix 2: src/compression.c compressionEnqueueCandidate()
====================================================================

Use-after-free caught by AddressSanitizer on the first PR-B CI run.
The eligibility predicate accepts encoding==RAW; a value currently
in transient-view state (R2.5.7) reads as RAW because val_ptr is
the per-iteration temp uncompressed sds. compressionEnqueueCandidate
would then capture job->src = temp_sds, which restoreTransientEntry
frees at beforeSleep — leaving the worker thread's job->src dangling
into freed memory.

Fix: gate the enqueue on !transientViewActive(value). Skipping the
enqueue is functionally harmless — a value in transient view is
already compressed (the original frame is saved in the side-map and
will be restored at beforeSleep). One line added at the top of
compressionEnqueueCandidate, with an explanatory comment naming the
exact ASan trace it fixes.

Tests: 392 gtests + 34 Tcl tests pass (28 unit + 6 integration).
Both BUILD_ZSTD={yes,no} build clean with -Werror. Verified the
asan fix locally by rebuilding with -fsanitize=address and re-running
the integration suite — no use-after-free.

…ite fixes

Adds tests/integration/compression.tcl — first end-to-end exercise of
the merged S2.x stack against a real workload. Builds on top of PR-A
(#33)'s runtime dict-generation infrastructure: each test imports a
freshly-trained dict via tests/support/compression-helpers.tcl, then
exercises one specific behaviour of the hot path.

Six test cases:

  1. Write-path round trip.
     master=compression + sweeper=disabled. SET a compressible value,
     poll until OBJECT ENCODING reports "compressed", verify GET
     round-trips the original bytes through the read-path transient
     view (R2.5.7).

  2. Sweeper compresses pre-existing keys.
     master=off + populate 100 RAW keys, then flip to
     master=compression + sweeper=enabled. Wait for ALL keys
     compressed; verify EVERY value round-trips (no spot-checks).

  3. Decompression drain.
     Continuing from #2's compressed state, flip to
     master=decompression + sweeper=enabled. Wait for ALL keys
     drained back to RAW; verify EVERY value round-trips.

  4. COMPRESSION SWEEP FORCE end-to-end.
     master=compression + sweeper=disabled (manual-only). Populate
     uncompressed, then run COMPRESSION SWEEP FORCE. Wait for ALL
     keys to be compressed by a single forced pass; verify EVERY
     value round-trips.

  5. Mixed workload preserves data integrity under live sweeper.
     master=compression + sweeper=enabled + 50% pacing. Populate 200
     keys, run 500 random ops (GET/SET/APPEND/SETRANGE/DEL).

  6. Ineligibility — values outside the size envelope and hot keys.
     Verifies the eligibility predicate (R2.2): values below
     compression-min-value-size, values above compression-max-value-size,
     and freshly-written hot keys must NOT be compressed even with the
     sweeper running at maximum cadence.

Prerequisite fix 1: src/object.c strEncoding()
================================================

OBJECT ENCODING was returning "unknown" for compressed values
because strEncoding() didn't have a case for OBJ_ENCODING_COMPRESSED.
Design R2.7.1 requires it returns "compressed". One-line fix that
slipped through earlier S2 PRs.

Prerequisite fix 2: src/compression.c compressionEnqueueCandidate()
====================================================================

Use-after-free caught by AddressSanitizer on the first PR-B CI run.
The eligibility predicate accepts encoding==RAW; a value currently
in transient-view state (R2.5.7) reads as RAW because val_ptr is
the per-iteration temp uncompressed sds. compressionEnqueueCandidate
would then capture job->src = temp_sds, which restoreTransientEntry
frees at beforeSleep — leaving the worker thread's job->src dangling
into freed memory.

Fix: gate the enqueue on !transientViewActive(value). Skipping the
enqueue is functionally harmless — a value in transient view is
already compressed (the original frame is saved in the side-map and
will be restored at beforeSleep). One line added at the top of
compressionEnqueueCandidate, with an explanatory comment naming the
exact ASan trace it fixes.

Tests: 392 gtests + 34 Tcl tests pass (28 unit + 6 integration).
Both BUILD_ZSTD={yes,no} build clean with -Werror. Verified the
asan fix locally by rebuilding with -fsanitize=address and re-running
the integration suite — no use-after-free.

… fix (#34)

* [Topic-2 PR-A] COMPRESSION DICT-IMPORT + runtime dict-generation test infra (R2.3.10) (#33)

Implements the minimal preshared-dictionary import surface so
integration tests can run before S1.x training lands. R2.3.10 + §4.5
in the design doc: operator base64-encodes a ZSTD-trained dictionary
and installs it via `COMPRESSION DICT-IMPORT <base64-bytes>`. The
new dict is promoted as active; the previous active is retired
through the existing registry path.

The blocker this solves: `compressionEnqueueCandidate` early-returns
when `compressionRegistryActive() == NULL`, so without a trained
dict the entire write/sweep path is a no-op. Tests that exercise
end-to-end compression behaviour (S2.7 write hook, S2.8 read hook,
S2.9 sweeper) need a way to install a dict; this PR is that way.
S1.x's full training implementation (BIO_COMPRESSION_TRAIN +
ZDICT_trainFromBuffer on the bio thread) lands separately on the
@GilboaAWS track.

Implementation
--------------

Hyphenated subcommand `DICT-IMPORT` (CLUSTER COUNT-FAILURE-REPORTS
precedent — RESP doesn't have nested subcommand containers).

Validation, in order:
  - Base64 decoding (private static `base64Decode` in compression.c;
    standard alphabet, optional `=` padding, whitespace rejected).
  - 4-byte ZSTD magic 0xEC30A437 — rejects raw-content prefixes and
    other non-trained bytes. Exotic operators with raw prefixes will
    have to find another route; the 99% case is "I trained a dict,
    I'm importing it" and that case wants real validation.
  - `ZSTD_createCDict` / `ZSTD_createDDict` — these accept arbitrary
    bytes as raw prefixes (never return NULL on garbage), so the
    magic check above is the actual content-validity gate. The ZSTD
    calls remain as belt-and-suspenders for OOM and similar.
  - `compressionRegistryAdd(pair, promote=1)` — same path a trained
    dict will use once S1.x lands.

Reply: integer dict_id on success, RESP error on rejection.

INFO renderer
-------------

Replaced the `compression_active_dict_id:0` and `compression_known_
dicts:0` placeholders with live values via the new registry accessor
`compressionRegistryGetKnownCount()`. Operators running this server
can now see imported dicts immediately in `INFO compression` /
`COMPRESSION STATUS`. Other field placeholders (compressed_objects,
ratio, etc.) remain at 0 until later S2 PRs land their counters.

Test infrastructure: runtime dict generation
--------------------------------------------

Static dict fixtures don't scale to the test matrix the project
needs (per-shape dicts for JSON / kv / log workloads, drift testing
where the dict was trained on shape A but workload arrives as shape
B, retraining cycles where dict A is replaced by dict B). Shipping
multiple ~10 KiB binaries under tests/assets/ would bloat the repo
and still not cover the drift case. Instead, we generate dicts at
test time, on demand, parametrized by data shape.

The dict generator MUST be external to valkey-server. If we used a
server-side test command (e.g. DEBUG COMPRESSION TRAIN-FROM-BYTES),
a bug in the server's training plumbing could mask itself — both
the test fixture and the production training path would share code
and exhibit the same bug. The infrastructure landed here uses a
separate process that calls only ZDICT_trainFromBuffer directly:

  - tests/helpers/gen-zstd-dict.c (new):
    Standalone helper binary. Reads samples from stdin in a simple
    binary protocol (4-byte big-endian length + N bytes per sample,
    repeated until EOF), trains a ZSTD dictionary via
    ZDICT_trainFromBuffer, writes the trained dict to a path passed
    on argv. Links against the same vendored deps/zstd/libzstd.a as
    valkey-server, so ZDICT API behaviour matches what production
    will use, but runs in a separate process with no shared memory
    or globals with the SUT.

  - src/Makefile:
    Adds tests/helpers/gen-zstd-dict to ALL_BUILD_PREREQUISITES when
    BUILD_ZSTD=yes (gated by the same ifeq block that controls the
    feature itself). BUILD_ZSTD=no skips it — there's no compression
    feature to test. clean target updated.

  - tests/support/compression-helpers.tcl (new):
    Sample generators (gen_kv_samples, gen_json_samples,
    gen_log_samples) producing reproducible per-seed sample lists.
    gen_drifted_samples mixes two shapes by a `drift` fraction in
    [0,1] for drift / retraining tests. train_dict_from_samples
    pipes samples through the helper binary; import_dict is the
    convenience wrapper that trains + base64-encodes + sends
    COMPRESSION DICT-IMPORT.

  - tests/test_helper.tcl: source the new support file.

  - tests/unit/type/compression.tcl: the existing "import a real
    trained dict" Tcl test now generates samples + trains at test
    time instead of reading a static fixture. New "drift mixer
    sanity" test verifies the gen_drifted_samples helper itself.

  - tests/assets/test-compression.dict: deleted (no longer needed).

Tests
-----

gtest (392 total, +1 new under CompressionRegistryTest):
  - GetKnownCountTracksAddsAndCapEnforcement — verifies the new
    accessor moves with each Add and stays at the cap on rejection.

Tcl (28 total, +6 new under unit/type/compression):
  - Rejects malformed base64.
  - Rejects valid base64 without ZSTD magic ("hello world").
  - Rejects payloads smaller than the magic header.
  - Validates arity at the command-table level.
  - Imports a real runtime-trained dict, verifies INFO reflects
    active_dict_id+known_dicts, second import promotes new + retires
    previous (count=2).
  - Smoke-tests gen_drifted_samples (verifies pure-A / pure-B / 50-50
    mixing produce shape-distinct outputs).

Verification
------------

  - 392 gtests pass (was 391 — +1 new).
  - 28 Tcl tests pass (was 22 — +6 new).
  - BUILD_ZSTD=yes and BUILD_ZSTD=no both clean with -Werror.
  - gen-zstd-dict helper builds only when BUILD_ZSTD=yes and is
    invoked correctly by the Tcl wrapper end-to-end.

Out of scope for this PR
------------------------

  - DICT-EXPORT (R2.3.10 mentions both; symmetric implementation is
    a small follow-up once we have one operator who needs it).
  - DICT-LIST / DICT-DROP (§4.5; pending S4.x observability work).
  - Real training (S1.x — @GilboaAWS track).
  - Topic-2 PR-B (compression-stress.tcl) — the integration stress
    test that USES this command. Lands next.

* [Topic-2 PR-B] compression-stress.tcl integration tests + 2 prerequisite fixes

Adds tests/integration/compression.tcl — first end-to-end exercise of
the merged S2.x stack against a real workload. Builds on top of PR-A
(#33)'s runtime dict-generation infrastructure: each test imports a
freshly-trained dict via tests/support/compression-helpers.tcl, then
exercises one specific behaviour of the hot path.

Six test cases:

  1. Write-path round trip.
     master=compression + sweeper=disabled. SET a compressible value,
     poll until OBJECT ENCODING reports "compressed", verify GET
     round-trips the original bytes through the read-path transient
     view (R2.5.7).

  2. Sweeper compresses pre-existing keys.
     master=off + populate 100 RAW keys, then flip to
     master=compression + sweeper=enabled. Wait for ALL keys
     compressed; verify EVERY value round-trips (no spot-checks).

  3. Decompression drain.
     Continuing from #2's compressed state, flip to
     master=decompression + sweeper=enabled. Wait for ALL keys
     drained back to RAW; verify EVERY value round-trips.

  4. COMPRESSION SWEEP FORCE end-to-end.
     master=compression + sweeper=disabled (manual-only). Populate
     uncompressed, then run COMPRESSION SWEEP FORCE. Wait for ALL
     keys to be compressed by a single forced pass; verify EVERY
     value round-trips.

  5. Mixed workload preserves data integrity under live sweeper.
     master=compression + sweeper=enabled + 50% pacing. Populate 200
     keys, run 500 random ops (GET/SET/APPEND/SETRANGE/DEL).

  6. Ineligibility — values outside the size envelope and hot keys.
     Verifies the eligibility predicate (R2.2): values below
     compression-min-value-size, values above compression-max-value-size,
     and freshly-written hot keys must NOT be compressed even with the
     sweeper running at maximum cadence.

Prerequisite fix 1: src/object.c strEncoding()
================================================

OBJECT ENCODING was returning "unknown" for compressed values
because strEncoding() didn't have a case for OBJ_ENCODING_COMPRESSED.
Design R2.7.1 requires it returns "compressed". One-line fix that
slipped through earlier S2 PRs.

Prerequisite fix 2: src/compression.c compressionEnqueueCandidate()
====================================================================

Use-after-free caught by AddressSanitizer on the first PR-B CI run.
The eligibility predicate accepts encoding==RAW; a value currently
in transient-view state (R2.5.7) reads as RAW because val_ptr is
the per-iteration temp uncompressed sds. compressionEnqueueCandidate
would then capture job->src = temp_sds, which restoreTransientEntry
frees at beforeSleep — leaving the worker thread's job->src dangling
into freed memory.

Fix: gate the enqueue on !transientViewActive(value). Skipping the
enqueue is functionally harmless — a value in transient view is
already compressed (the original frame is saved in the side-map and
will be restored at beforeSleep). One line added at the top of
compressionEnqueueCandidate, with an explanatory comment naming the
exact ASan trace it fixes.

Tests: 392 gtests + 34 Tcl tests pass (28 unit + 6 integration).
Both BUILD_ZSTD={yes,no} build clean with -Werror. Verified the
asan fix locally by rebuilding with -fsanitize=address and re-running
the integration suite — no use-after-free.

* [Topic-2 PR-A] COMPRESSION DICT-IMPORT + runtime dict-generation test infra (R2.3.10)

Implements the minimal preshared-dictionary import surface so
integration tests can run before S1.x training lands. R2.3.10 + §4.5
in the design doc: operator base64-encodes a ZSTD-trained dictionary
and installs it via `COMPRESSION DICT-IMPORT <base64-bytes>`. The
new dict is promoted as active; the previous active is retired
through the existing registry path.

The blocker this solves: `compressionEnqueueCandidate` early-returns
when `compressionRegistryActive() == NULL`, so without a trained
dict the entire write/sweep path is a no-op. Tests that exercise
end-to-end compression behaviour (S2.7 write hook, S2.8 read hook,
S2.9 sweeper) need a way to install a dict; this PR is that way.
S1.x's full training implementation (BIO_COMPRESSION_TRAIN +
ZDICT_trainFromBuffer on the bio thread) lands separately on the
@GilboaAWS track.

Implementation
--------------

Hyphenated subcommand `DICT-IMPORT` (CLUSTER COUNT-FAILURE-REPORTS
precedent — RESP doesn't have nested subcommand containers).

Validation, in order:
  - Base64 decoding (private static `base64Decode` in compression.c;
    standard alphabet, optional `=` padding, whitespace rejected).
  - 4-byte ZSTD magic 0xEC30A437 — rejects raw-content prefixes and
    other non-trained bytes. Exotic operators with raw prefixes will
    have to find another route; the 99% case is "I trained a dict,
    I'm importing it" and that case wants real validation.
  - `ZSTD_createCDict` / `ZSTD_createDDict` — these accept arbitrary
    bytes as raw prefixes (never return NULL on garbage), so the
    magic check above is the actual content-validity gate. The ZSTD
    calls remain as belt-and-suspenders for OOM and similar.
  - `compressionRegistryAdd(pair, promote=1)` — same path a trained
    dict will use once S1.x lands.

Reply: integer dict_id on success, RESP error on rejection.

INFO renderer
-------------

Replaced the `compression_active_dict_id:0` and `compression_known_
dicts:0` placeholders with live values via the new registry accessor
`compressionRegistryGetKnownCount()`. Operators running this server
can now see imported dicts immediately in `INFO compression` /
`COMPRESSION STATUS`. Other field placeholders (compressed_objects,
ratio, etc.) remain at 0 until later S2 PRs land their counters.

Test infrastructure: runtime dict generation
--------------------------------------------

Static dict fixtures don't scale to the test matrix the project
needs (per-shape dicts for JSON / kv / log workloads, drift testing
where the dict was trained on shape A but workload arrives as shape
B, retraining cycles where dict A is replaced by dict B). Shipping
multiple ~10 KiB binaries under tests/assets/ would bloat the repo
and still not cover the drift case. Instead, we generate dicts at
test time, on demand, parametrized by data shape.

The dict generator MUST be external to valkey-server. If we used a
server-side test command (e.g. DEBUG COMPRESSION TRAIN-FROM-BYTES),
a bug in the server's training plumbing could mask itself — both
the test fixture and the production training path would share code
and exhibit the same bug. The infrastructure landed here uses a
separate process that calls only ZDICT_trainFromBuffer directly:

  - tests/helpers/gen-zstd-dict.c (new):
    Standalone helper binary. Reads samples from stdin in a simple
    binary protocol (4-byte big-endian length + N bytes per sample,
    repeated until EOF), trains a ZSTD dictionary via
    ZDICT_trainFromBuffer, writes the trained dict to a path passed
    on argv. Links against the same vendored deps/zstd/libzstd.a as
    valkey-server, so ZDICT API behaviour matches what production
    will use, but runs in a separate process with no shared memory
    or globals with the SUT.

  - src/Makefile:
    Adds tests/helpers/gen-zstd-dict to ALL_BUILD_PREREQUISITES when
    BUILD_ZSTD=yes (gated by the same ifeq block that controls the
    feature itself). BUILD_ZSTD=no skips it — there's no compression
    feature to test. clean target updated.

  - tests/support/compression-helpers.tcl (new):
    Sample generators (gen_kv_samples, gen_json_samples,
    gen_log_samples) producing reproducible per-seed sample lists.
    gen_drifted_samples mixes two shapes by a `drift` fraction in
    [0,1] for drift / retraining tests. train_dict_from_samples
    pipes samples through the helper binary; import_dict is the
    convenience wrapper that trains + base64-encodes + sends
    COMPRESSION DICT-IMPORT.

  - tests/test_helper.tcl: source the new support file.

  - tests/unit/type/compression.tcl: the existing "import a real
    trained dict" Tcl test now generates samples + trains at test
    time instead of reading a static fixture. New "drift mixer
    sanity" test verifies the gen_drifted_samples helper itself.

  - tests/assets/test-compression.dict: deleted (no longer needed).

Tests
-----

gtest (392 total, +1 new under CompressionRegistryTest):
  - GetKnownCountTracksAddsAndCapEnforcement — verifies the new
    accessor moves with each Add and stays at the cap on rejection.

Tcl (28 total, +6 new under unit/type/compression):
  - Rejects malformed base64.
  - Rejects valid base64 without ZSTD magic ("hello world").
  - Rejects payloads smaller than the magic header.
  - Validates arity at the command-table level.
  - Imports a real runtime-trained dict, verifies INFO reflects
    active_dict_id+known_dicts, second import promotes new + retires
    previous (count=2).
  - Smoke-tests gen_drifted_samples (verifies pure-A / pure-B / 50-50
    mixing produce shape-distinct outputs).

Verification
------------

  - 392 gtests pass (was 391 — +1 new).
  - 28 Tcl tests pass (was 22 — +6 new).
  - BUILD_ZSTD=yes and BUILD_ZSTD=no both clean with -Werror.
  - gen-zstd-dict helper builds only when BUILD_ZSTD=yes and is
    invoked correctly by the Tcl wrapper end-to-end.

Out of scope for this PR
------------------------

  - DICT-EXPORT (R2.3.10 mentions both; symmetric implementation is
    a small follow-up once we have one operator who needs it).
  - DICT-LIST / DICT-DROP (§4.5; pending S4.x observability work).
  - Real training (S1.x — @GilboaAWS track).
  - Topic-2 PR-B (compression-stress.tcl) — the integration stress
    test that USES this command. Lands next.

* [Topic-2 PR-B] compression-stress.tcl integration tests + strEncoding fix (#34)

* [Topic-2 PR-A] COMPRESSION DICT-IMPORT + runtime dict-generation test infra (R2.3.10) (#33)

Implements the minimal preshared-dictionary import surface so
integration tests can run before S1.x training lands. R2.3.10 + §4.5
in the design doc: operator base64-encodes a ZSTD-trained dictionary
and installs it via `COMPRESSION DICT-IMPORT <base64-bytes>`. The
new dict is promoted as active; the previous active is retired
through the existing registry path.

The blocker this solves: `compressionEnqueueCandidate` early-returns
when `compressionRegistryActive() == NULL`, so without a trained
dict the entire write/sweep path is a no-op. Tests that exercise
end-to-end compression behaviour (S2.7 write hook, S2.8 read hook,
S2.9 sweeper) need a way to install a dict; this PR is that way.
S1.x's full training implementation (BIO_COMPRESSION_TRAIN +
ZDICT_trainFromBuffer on the bio thread) lands separately on the
@GilboaAWS track.

Implementation
--------------

Hyphenated subcommand `DICT-IMPORT` (CLUSTER COUNT-FAILURE-REPORTS
precedent — RESP doesn't have nested subcommand containers).

Validation, in order:
  - Base64 decoding (private static `base64Decode` in compression.c;
    standard alphabet, optional `=` padding, whitespace rejected).
  - 4-byte ZSTD magic 0xEC30A437 — rejects raw-content prefixes and
    other non-trained bytes. Exotic operators with raw prefixes will
    have to find another route; the 99% case is "I trained a dict,
    I'm importing it" and that case wants real validation.
  - `ZSTD_createCDict` / `ZSTD_createDDict` — these accept arbitrary
    bytes as raw prefixes (never return NULL on garbage), so the
    magic check above is the actual content-validity gate. The ZSTD
    calls remain as belt-and-suspenders for OOM and similar.
  - `compressionRegistryAdd(pair, promote=1)` — same path a trained
    dict will use once S1.x lands.

Reply: integer dict_id on success, RESP error on rejection.

INFO renderer
-------------

Replaced the `compression_active_dict_id:0` and `compression_known_
dicts:0` placeholders with live values via the new registry accessor
`compressionRegistryGetKnownCount()`. Operators running this server
can now see imported dicts immediately in `INFO compression` /
`COMPRESSION STATUS`. Other field placeholders (compressed_objects,
ratio, etc.) remain at 0 until later S2 PRs land their counters.

Test infrastructure: runtime dict generation
--------------------------------------------

Static dict fixtures don't scale to the test matrix the project
needs (per-shape dicts for JSON / kv / log workloads, drift testing
where the dict was trained on shape A but workload arrives as shape
B, retraining cycles where dict A is replaced by dict B). Shipping
multiple ~10 KiB binaries under tests/assets/ would bloat the repo
and still not cover the drift case. Instead, we generate dicts at
test time, on demand, parametrized by data shape.

The dict generator MUST be external to valkey-server. If we used a
server-side test command (e.g. DEBUG COMPRESSION TRAIN-FROM-BYTES),
a bug in the server's training plumbing could mask itself — both
the test fixture and the production training path would share code
and exhibit the same bug. The infrastructure landed here uses a
separate process that calls only ZDICT_trainFromBuffer directly:

  - tests/helpers/gen-zstd-dict.c (new):
    Standalone helper binary. Reads samples from stdin in a simple
    binary protocol (4-byte big-endian length + N bytes per sample,
    repeated until EOF), trains a ZSTD dictionary via
    ZDICT_trainFromBuffer, writes the trained dict to a path passed
    on argv. Links against the same vendored deps/zstd/libzstd.a as
    valkey-server, so ZDICT API behaviour matches what production
    will use, but runs in a separate process with no shared memory
    or globals with the SUT.

  - src/Makefile:
    Adds tests/helpers/gen-zstd-dict to ALL_BUILD_PREREQUISITES when
    BUILD_ZSTD=yes (gated by the same ifeq block that controls the
    feature itself). BUILD_ZSTD=no skips it — there's no compression
    feature to test. clean target updated.

  - tests/support/compression-helpers.tcl (new):
    Sample generators (gen_kv_samples, gen_json_samples,
    gen_log_samples) producing reproducible per-seed sample lists.
    gen_drifted_samples mixes two shapes by a `drift` fraction in
    [0,1] for drift / retraining tests. train_dict_from_samples
    pipes samples through the helper binary; import_dict is the
    convenience wrapper that trains + base64-encodes + sends
    COMPRESSION DICT-IMPORT.

  - tests/test_helper.tcl: source the new support file.

  - tests/unit/type/compression.tcl: the existing "import a real
    trained dict" Tcl test now generates samples + trains at test
    time instead of reading a static fixture. New "drift mixer
    sanity" test verifies the gen_drifted_samples helper itself.

  - tests/assets/test-compression.dict: deleted (no longer needed).

Tests
-----

gtest (392 total, +1 new under CompressionRegistryTest):
  - GetKnownCountTracksAddsAndCapEnforcement — verifies the new
    accessor moves with each Add and stays at the cap on rejection.

Tcl (28 total, +6 new under unit/type/compression):
  - Rejects malformed base64.
  - Rejects valid base64 without ZSTD magic ("hello world").
  - Rejects payloads smaller than the magic header.
  - Validates arity at the command-table level.
  - Imports a real runtime-trained dict, verifies INFO reflects
    active_dict_id+known_dicts, second import promotes new + retires
    previous (count=2).
  - Smoke-tests gen_drifted_samples (verifies pure-A / pure-B / 50-50
    mixing produce shape-distinct outputs).

Verification
------------

  - 392 gtests pass (was 391 — +1 new).
  - 28 Tcl tests pass (was 22 — +6 new).
  - BUILD_ZSTD=yes and BUILD_ZSTD=no both clean with -Werror.
  - gen-zstd-dict helper builds only when BUILD_ZSTD=yes and is
    invoked correctly by the Tcl wrapper end-to-end.

Out of scope for this PR
------------------------

  - DICT-EXPORT (R2.3.10 mentions both; symmetric implementation is
    a small follow-up once we have one operator who needs it).
  - DICT-LIST / DICT-DROP (§4.5; pending S4.x observability work).
  - Real training (S1.x — @GilboaAWS track).
  - Topic-2 PR-B (compression-stress.tcl) — the integration stress
    test that USES this command. Lands next.

* [Topic-2 PR-B] compression-stress.tcl integration tests + 2 prerequisite fixes

Adds tests/integration/compression.tcl — first end-to-end exercise of
the merged S2.x stack against a real workload. Builds on top of PR-A
(#33)'s runtime dict-generation infrastructure: each test imports a
freshly-trained dict via tests/support/compression-helpers.tcl, then
exercises one specific behaviour of the hot path.

Six test cases:

  1. Write-path round trip.
     master=compression + sweeper=disabled. SET a compressible value,
     poll until OBJECT ENCODING reports "compressed", verify GET
     round-trips the original bytes through the read-path transient
     view (R2.5.7).

  2. Sweeper compresses pre-existing keys.
     master=off + populate 100 RAW keys, then flip to
     master=compression + sweeper=enabled. Wait for ALL keys
     compressed; verify EVERY value round-trips (no spot-checks).

  3. Decompression drain.
     Continuing from #2's compressed state, flip to
     master=decompression + sweeper=enabled. Wait for ALL keys
     drained back to RAW; verify EVERY value round-trips.

  4. COMPRESSION SWEEP FORCE end-to-end.
     master=compression + sweeper=disabled (manual-only). Populate
     uncompressed, then run COMPRESSION SWEEP FORCE. Wait for ALL
     keys to be compressed by a single forced pass; verify EVERY
     value round-trips.

  5. Mixed workload preserves data integrity under live sweeper.
     master=compression + sweeper=enabled + 50% pacing. Populate 200
     keys, run 500 random ops (GET/SET/APPEND/SETRANGE/DEL).

  6. Ineligibility — values outside the size envelope and hot keys.
     Verifies the eligibility predicate (R2.2): values below
     compression-min-value-size, values above compression-max-value-size,
     and freshly-written hot keys must NOT be compressed even with the
     sweeper running at maximum cadence.

Prerequisite fix 1: src/object.c strEncoding()
================================================

OBJECT ENCODING was returning "unknown" for compressed values
because strEncoding() didn't have a case for OBJ_ENCODING_COMPRESSED.
Design R2.7.1 requires it returns "compressed". One-line fix that
slipped through earlier S2 PRs.

Prerequisite fix 2: src/compression.c compressionEnqueueCandidate()
====================================================================

Use-after-free caught by AddressSanitizer on the first PR-B CI run.
The eligibility predicate accepts encoding==RAW; a value currently
in transient-view state (R2.5.7) reads as RAW because val_ptr is
the per-iteration temp uncompressed sds. compressionEnqueueCandidate
would then capture job->src = temp_sds, which restoreTransientEntry
frees at beforeSleep — leaving the worker thread's job->src dangling
into freed memory.

Fix: gate the enqueue on !transientViewActive(value). Skipping the
enqueue is functionally harmless — a value in transient view is
already compressed (the original frame is saved in the side-map and
will be restored at beforeSleep). One line added at the top of
compressionEnqueueCandidate, with an explanatory comment naming the
exact ASan trace it fixes.

Tests: 392 gtests + 34 Tcl tests pass (28 unit + 6 integration).
Both BUILD_ZSTD={yes,no} build clean with -Werror. Verified the
asan fix locally by rebuilding with -fsanitize=address and re-running
the integration suite — no use-after-free.