Skip to content

[backport] Backport sweep for 9.1#3733

Closed
valkeyrie-ops[bot] wants to merge 7 commits into
9.1from
agent/backport/sweep/9.1
Closed

[backport] Backport sweep for 9.1#3733
valkeyrie-ops[bot] wants to merge 7 commits into
9.1from
agent/backport/sweep/9.1

Conversation

@valkeyrie-ops

@valkeyrie-ops valkeyrie-ops Bot commented May 16, 2026

Copy link
Copy Markdown

Backport sweep for 9.1

Automated cherry-picks from PRs marked "To be backported".

Applied

Source PR Title Detail
#3380 CLUSTERSCAN MATCH pattern maps to a specific slot optimizations already on backport branch
#3619 Fix invalid memory access in RESTORE with malformed zipmap (CVE-2026-25243) already on backport branch
#3654 Use full hash-seed bytes when deriving SipHash seed already on backport branch
#3678 Restore ERR prefix and error-code preservation in Lua scripting
#3719 Fix deferred freeClient clobbering replication state after replicaof already on backport branch
#3743 Fix buffered_reply assert in HFE commands with module keyspace notifications
#3753 Update the datasize to match the automated perfs

Generated by valkey-ci-agent using Claude Code.

@codecov

codecov Bot commented May 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.63%. Comparing base (a4a776f) to head (073a6fa).

Additional details and impacted files
@@            Coverage Diff             @@
##              9.1    #3733      +/-   ##
==========================================
+ Coverage   76.52%   76.63%   +0.10%     
==========================================
  Files         163      163              
  Lines       80991    81009      +18     
==========================================
+ Hits        61980    62079      +99     
+ Misses      19011    18930      -81     
Files with missing lines Coverage Δ
src/cluster.c 92.66% <ø> (ø)
src/debug.c 54.95% <100.00%> (+0.11%) ⬆️
src/networking.c 92.79% <100.00%> (+0.41%) ⬆️
src/replication.c 86.26% <100.00%> (+0.05%) ⬆️
src/server.c 89.45% <100.00%> (+0.03%) ⬆️
src/server.h 100.00% <ø> (ø)
src/t_hash.c 95.42% <100.00%> (-0.06%) ⬇️
src/tls.c 17.64% <ø> (ø)
src/util.c 69.94% <100.00%> (+0.25%) ⬆️

... and 19 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@madolson

Copy link
Copy Markdown
Member

@sarthakaggarwal97 There seems to be a bunch of sentinel tests that are exploding. Is there any way to ask claude to automatically try to fix it?

@coderabbitai

coderabbitai Bot commented May 18, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 72e3091b-7f1f-4a34-baf7-22673e00a562

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands and usage tips.

@sarthakaggarwal97

sarthakaggarwal97 commented May 18, 2026

Copy link
Copy Markdown
Contributor

@sarthakaggarwal97 There seems to be a bunch of sentinel tests that are exploding. Is there any way to ask claude to automatically try to fix it?

These tests are unrelated to the PR. This was a follow up item to try and fix CI / Weekly / Daily Tests. Let me check if I can do something about it.

nmvk and others added 7 commits May 18, 2026 23:10
When a MATCH pattern maps to a specific slot, `CLUSTERSCAN` can skip
directly to that slot
instead of walking through all slots one by one.

- On `cursor 0`, starts directly at the matching slot
- If cursor is behind the matching slot, jumps forward
- If cursor is ahead of the matching slot, we conclude the scan as we
cannot match keys
- If both SLOT and MATCH are provided but target different slots,
returns 0 immediately

Signed-off-by: nmvk <r@nmvk.com>
…25243) (#3619)

Root cause: zipmapValidateIntegrity() and zipmapNext() use different
methods to calculate pointer advancement for length-encoded fields.
Validation reads the actual encoded size via
zipmapGetEncodedLengthSize() (which returns 5 for the 0xFE prefix), but
zipmapRawKeyLength() (used by zipmapNext during hash conversion)
recalculates via zipmapEncodeLength() which returns 1 for decoded
lengths < 254. A crafted zipmap with an overlong 5-byte encoding for a
small length passes validation but causes a 4-byte pointer mismatch in
zipmapNext(), leading to heap buffer over-reads during the
zipmap-to-listpack conversion.

Fix: add sanity checks in zipmapValidateIntegrity() to reject entries
where the decoded length < ZIPMAP_BIGLEN (254) but the encoding uses
more than 1 byte. This is applied to both field-name and value lengths.

Test: added a regression test in tests/unit/dump.tcl that crafts a
RESTORE payload with a 2-entry zipmap where the first field uses an
overlong 5-byte length encoding for value 3. Post-patch, this is cleanly
rejected by zipmapValidateIntegrity(). Pre-patch, the misaligned
zipmapNext() reads garbage (confirmed via server log: "Hash zipmap with
dup elements, or big length (0)") which also produces an error, so the
test serves as a defense-in-depth regression anchor rather than a strict
pass/fail differentiator. The actual heap over-read is detectable with
AddressSanitizer builds.

Signed-off-by: ikolomi <ikolomin@amazon.com>
Co-authored-by: ikolomi <ikolomin@amazon.com>
…3719)

## Summary

This PR addresses 2 race conditions where deferred `freeClient`
(introduced by #3324) clobbers replication state set by `REPLICAOF`
commands.

Found while triaging the recurring `test-ubuntu-tls-io-threads` daily CI
failure in `tests/unit/wait.tcl`.

## Root Cause

Since PR #3324 ("Redesign IO threading communication model"),
`freeClient()` on a primary client with pending IO is deferred via
`freeClientAsync` (gated on `clientHasPendingIO`). When the deferred
free eventually executes, it chains through `replicationCachePrimary()`
-> `replicationHandlePrimaryDisconnection()`, which unconditionally sets
`server.repl_state = REPL_STATE_CONNECT`.

This causes two bugs:

### Bug 1: REPLICAOF NO ONE (SIGSEGV)

`replicationUnsetPrimary()` sets `primary_host = NULL` before calling
`freeClient`. The deferred free runs later, sets `repl_state =
REPL_STATE_CONNECT` while `primary_host` is still NULL.
`replicationCron` then calls `connectWithPrimary()` which passes NULL to
`connTLSConnect()` -> `inet_pton(AF_INET, NULL, ...)` -> SIGSEGV.

### Bug 2: REPLICAOF newhost newport (connection leak)

`replicationSetPrimary()` calls `freeClient(old_primary)` (deferred),
then sets `primary_host` to the new IP and progresses `repl_state` to
`REPL_STATE_CONNECTING` with a new connection handle in
`server.repl_transfer_s`. The deferred free runs later, clobbers
`repl_state` back to `REPL_STATE_CONNECT`. `replicationCron` then calls
`connectWithPrimary()` again, overwriting `server.repl_transfer_s`
without closing the previous connection -- an FD leak.

## Fix

Make `replicationHandlePrimaryDisconnection()` only transition to
`REPL_STATE_CONNECT` when `repl_state` is still `REPL_STATE_CONNECTED`
and `primary_host` is set. This means the disconnection is genuine and
no other state transition has already occurred. If `repl_state` has
already moved on (CONNECT, CONNECTING, NONE, etc.), the deferred free is
stale and the function leaves the state untouched.

Additionally:
- `connTLSConnect()`: Return `C_ERR` if `addr` is NULL (defense in
depth).
- `tests/unit/wait.tcl`: Add 10s timeout to the blocking WAITAOF test,
and add dedicated tests for the repoint scenario.

## Reproduction

Reproduced locally by establishing TLS replication and executing
`REPLICAOF NO ONE`:
- Without fix: server crashes with signal 11, accessing address 0x0
- With fix: server continues operating normally

## Testing

- Full `unit/wait` test suite passes (51/51) with IO threads enabled
- New tests "Repoint replica between primaries does not leak connections
or crash" and "Rapid repoint does not crash or leak" pass
- Crash reproduced locally over TLS (SIGSEGV without fix, graceful
handling with fix)

---------

Signed-off-by: Yaron Sananes <yaron.sananes@gmail.com>
Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
The hash-seed config is an sds string and may contain embedded NUL
bytes (sdssplitargs preserves \xNN escapes inside double quotes).

In the old code getHashSeedFromString() used strlen() on it, so two
hash-seed values differing only after a \x00 byte collapsed to the
same SipHash seed, can break it.

hash-seed was added in #2608.

---------

Signed-off-by: Binbin <binloveplay1314@qq.com>
Closes #3663.

## Why

Three callers in `src/modules/lua/script_lua.c` lost error-code handling
during the #2858 move of the Lua scripting engine into a module. On the
wire, `redis.pcall()` (no args), `redis.setresp(4)`, `pcall(redis.log,
1)`, `pcall(redis.log, 10, 'msg')`, and `pcall(redis.set_repl)` now
return the bare message instead of `ERR <message>`. OOM and WRONGTYPE
happened to keep working because their code letters survived elsewhere
by accident.

Reproducer:

```
$ ./src/valkey-cli eval "return redis.pcall()" 0
"Please specify at least one argument for this redis lib call"   # before this PR
"ERR Please specify at least one argument for this redis lib call"  # with this PR
```

## What changed

In `src/modules/lua/script_lua.c`:

1. `luaPushErrorBuff` — bare-message branch (case 2): prepend `ERR `
unless the message already starts with a code letter (defensive — five
existing callers hardcode `"ERR ..."` strings; double-prefixing them
would also be wrong).
2. `luaProcessReplyError` push_error branch: re-add the leading `-`
before delegating to `luaPushError` so the case-1 code-extraction path
handles `OOM`, `WRONGTYPE`, `READONLY`, etc.
3. `errorCallback` errno==0 branch: same re-add-`-` approach.

41 LoC across the three functions.

## Test

`tests/unit/scripting.tcl` regression at line 1028 covers:

- All five direct-API errors above — asserts `ERR ` prefix
- WRONGTYPE error path — asserts the code survives and is not
double-prefixed
- Runs 4 times under the `foreach is_eval × foreach
script_compatibility_api` matrix

Fails on `unstable`, passes with this PR.

---------

Signed-off-by: 1fanwang <1fannnw@gmail.com>
Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
…cations (#3743)

When a module subscribes to NOTIFY_HASH keyspace events and blocks the
client in the notification callback, hexpireGenericCommand,
hgetdelCommand,
and hpersistCommand would trigger debugServerAssert in
notifyKeyspaceEvent()
because addReplyArrayLen() sets buffered_reply=1 before the notification
fires. This debug assert was added in #1819.

Affected commands: HEXPIRE, HPEXPIRE, HEXPIREAT, HPEXPIREAT, HGETDEL,
HPERSIST.

---------

Signed-off-by: Binbin <binloveplay1314@qq.com>
Update the data-size to match the automated perf benchmarking as 96 is
also embedded so moving to 128 to test for non-embedded.

I have also update the wfs to use the config files from the
`valkey-perf-benchmark` so we don't need to track the configs in 2
places, everything will be pulled from source `valkey-perf-benchmark`
and the configs will be standard throughout all runs

---------

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
@valkeyrie-ops valkeyrie-ops Bot force-pushed the agent/backport/sweep/9.1 branch from b4577d6 to 073a6fa Compare May 18, 2026 23:11
@valkeyrie-ops valkeyrie-ops Bot marked this pull request as ready for review May 18, 2026 23:11
@madolson

Copy link
Copy Markdown
Member

We need to remove the new code that seems to be causing an issue.

@sarthakaggarwal97

Copy link
Copy Markdown
Contributor

I will close this PR so it runs the workflow again cleanly.

@valkeyrie-ops valkeyrie-ops Bot deleted the agent/backport/sweep/9.1 branch May 19, 2026 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants