[backport] Backport sweep for 9.1#3733
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 9.1 #3733 +/- ##
==========================================
+ Coverage 76.52% 76.63% +0.10%
==========================================
Files 163 163
Lines 80991 81009 +18
==========================================
+ Hits 61980 62079 +99
+ Misses 19011 18930 -81
🚀 New features to boost your workflow:
|
|
@sarthakaggarwal97 There seems to be a bunch of sentinel tests that are exploding. Is there any way to ask claude to automatically try to fix it? |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
These tests are unrelated to the PR. This was a follow up item to try and fix CI / Weekly / Daily Tests. Let me check if I can do something about it. |
When a MATCH pattern maps to a specific slot, `CLUSTERSCAN` can skip directly to that slot instead of walking through all slots one by one. - On `cursor 0`, starts directly at the matching slot - If cursor is behind the matching slot, jumps forward - If cursor is ahead of the matching slot, we conclude the scan as we cannot match keys - If both SLOT and MATCH are provided but target different slots, returns 0 immediately Signed-off-by: nmvk <r@nmvk.com>
…25243) (#3619) Root cause: zipmapValidateIntegrity() and zipmapNext() use different methods to calculate pointer advancement for length-encoded fields. Validation reads the actual encoded size via zipmapGetEncodedLengthSize() (which returns 5 for the 0xFE prefix), but zipmapRawKeyLength() (used by zipmapNext during hash conversion) recalculates via zipmapEncodeLength() which returns 1 for decoded lengths < 254. A crafted zipmap with an overlong 5-byte encoding for a small length passes validation but causes a 4-byte pointer mismatch in zipmapNext(), leading to heap buffer over-reads during the zipmap-to-listpack conversion. Fix: add sanity checks in zipmapValidateIntegrity() to reject entries where the decoded length < ZIPMAP_BIGLEN (254) but the encoding uses more than 1 byte. This is applied to both field-name and value lengths. Test: added a regression test in tests/unit/dump.tcl that crafts a RESTORE payload with a 2-entry zipmap where the first field uses an overlong 5-byte length encoding for value 3. Post-patch, this is cleanly rejected by zipmapValidateIntegrity(). Pre-patch, the misaligned zipmapNext() reads garbage (confirmed via server log: "Hash zipmap with dup elements, or big length (0)") which also produces an error, so the test serves as a defense-in-depth regression anchor rather than a strict pass/fail differentiator. The actual heap over-read is detectable with AddressSanitizer builds. Signed-off-by: ikolomi <ikolomin@amazon.com> Co-authored-by: ikolomi <ikolomin@amazon.com>
…3719) ## Summary This PR addresses 2 race conditions where deferred `freeClient` (introduced by #3324) clobbers replication state set by `REPLICAOF` commands. Found while triaging the recurring `test-ubuntu-tls-io-threads` daily CI failure in `tests/unit/wait.tcl`. ## Root Cause Since PR #3324 ("Redesign IO threading communication model"), `freeClient()` on a primary client with pending IO is deferred via `freeClientAsync` (gated on `clientHasPendingIO`). When the deferred free eventually executes, it chains through `replicationCachePrimary()` -> `replicationHandlePrimaryDisconnection()`, which unconditionally sets `server.repl_state = REPL_STATE_CONNECT`. This causes two bugs: ### Bug 1: REPLICAOF NO ONE (SIGSEGV) `replicationUnsetPrimary()` sets `primary_host = NULL` before calling `freeClient`. The deferred free runs later, sets `repl_state = REPL_STATE_CONNECT` while `primary_host` is still NULL. `replicationCron` then calls `connectWithPrimary()` which passes NULL to `connTLSConnect()` -> `inet_pton(AF_INET, NULL, ...)` -> SIGSEGV. ### Bug 2: REPLICAOF newhost newport (connection leak) `replicationSetPrimary()` calls `freeClient(old_primary)` (deferred), then sets `primary_host` to the new IP and progresses `repl_state` to `REPL_STATE_CONNECTING` with a new connection handle in `server.repl_transfer_s`. The deferred free runs later, clobbers `repl_state` back to `REPL_STATE_CONNECT`. `replicationCron` then calls `connectWithPrimary()` again, overwriting `server.repl_transfer_s` without closing the previous connection -- an FD leak. ## Fix Make `replicationHandlePrimaryDisconnection()` only transition to `REPL_STATE_CONNECT` when `repl_state` is still `REPL_STATE_CONNECTED` and `primary_host` is set. This means the disconnection is genuine and no other state transition has already occurred. If `repl_state` has already moved on (CONNECT, CONNECTING, NONE, etc.), the deferred free is stale and the function leaves the state untouched. Additionally: - `connTLSConnect()`: Return `C_ERR` if `addr` is NULL (defense in depth). - `tests/unit/wait.tcl`: Add 10s timeout to the blocking WAITAOF test, and add dedicated tests for the repoint scenario. ## Reproduction Reproduced locally by establishing TLS replication and executing `REPLICAOF NO ONE`: - Without fix: server crashes with signal 11, accessing address 0x0 - With fix: server continues operating normally ## Testing - Full `unit/wait` test suite passes (51/51) with IO threads enabled - New tests "Repoint replica between primaries does not leak connections or crash" and "Rapid repoint does not crash or leak" pass - Crash reproduced locally over TLS (SIGSEGV without fix, graceful handling with fix) --------- Signed-off-by: Yaron Sananes <yaron.sananes@gmail.com> Signed-off-by: Ran Shidlansik <ranshid@amazon.com> Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
The hash-seed config is an sds string and may contain embedded NUL bytes (sdssplitargs preserves \xNN escapes inside double quotes). In the old code getHashSeedFromString() used strlen() on it, so two hash-seed values differing only after a \x00 byte collapsed to the same SipHash seed, can break it. hash-seed was added in #2608. --------- Signed-off-by: Binbin <binloveplay1314@qq.com>
Closes #3663. ## Why Three callers in `src/modules/lua/script_lua.c` lost error-code handling during the #2858 move of the Lua scripting engine into a module. On the wire, `redis.pcall()` (no args), `redis.setresp(4)`, `pcall(redis.log, 1)`, `pcall(redis.log, 10, 'msg')`, and `pcall(redis.set_repl)` now return the bare message instead of `ERR <message>`. OOM and WRONGTYPE happened to keep working because their code letters survived elsewhere by accident. Reproducer: ``` $ ./src/valkey-cli eval "return redis.pcall()" 0 "Please specify at least one argument for this redis lib call" # before this PR "ERR Please specify at least one argument for this redis lib call" # with this PR ``` ## What changed In `src/modules/lua/script_lua.c`: 1. `luaPushErrorBuff` — bare-message branch (case 2): prepend `ERR ` unless the message already starts with a code letter (defensive — five existing callers hardcode `"ERR ..."` strings; double-prefixing them would also be wrong). 2. `luaProcessReplyError` push_error branch: re-add the leading `-` before delegating to `luaPushError` so the case-1 code-extraction path handles `OOM`, `WRONGTYPE`, `READONLY`, etc. 3. `errorCallback` errno==0 branch: same re-add-`-` approach. 41 LoC across the three functions. ## Test `tests/unit/scripting.tcl` regression at line 1028 covers: - All five direct-API errors above — asserts `ERR ` prefix - WRONGTYPE error path — asserts the code survives and is not double-prefixed - Runs 4 times under the `foreach is_eval × foreach script_compatibility_api` matrix Fails on `unstable`, passes with this PR. --------- Signed-off-by: 1fanwang <1fannnw@gmail.com> Signed-off-by: Ran Shidlansik <ranshid@amazon.com> Co-authored-by: Ran Shidlansik <ranshid@amazon.com> Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
…cations (#3743) When a module subscribes to NOTIFY_HASH keyspace events and blocks the client in the notification callback, hexpireGenericCommand, hgetdelCommand, and hpersistCommand would trigger debugServerAssert in notifyKeyspaceEvent() because addReplyArrayLen() sets buffered_reply=1 before the notification fires. This debug assert was added in #1819. Affected commands: HEXPIRE, HPEXPIRE, HEXPIREAT, HPEXPIREAT, HGETDEL, HPERSIST. --------- Signed-off-by: Binbin <binloveplay1314@qq.com>
Update the data-size to match the automated perf benchmarking as 96 is also embedded so moving to 128 to test for non-embedded. I have also update the wfs to use the config files from the `valkey-perf-benchmark` so we don't need to track the configs in 2 places, everything will be pulled from source `valkey-perf-benchmark` and the configs will be standard throughout all runs --------- Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
b4577d6 to
073a6fa
Compare
|
We need to remove the new code that seems to be causing an issue. |
|
I will close this PR so it runs the workflow again cleanly. |
Backport sweep for 9.1
Automated cherry-picks from PRs marked "To be backported".
Applied
#3380#3619#3654#3678#3719#3743#3753Generated by valkey-ci-agent using Claude Code.