IO-Threads redesign cleanup work#3544
Merged
Merged
Conversation
Signed-off-by: akash kumar <akumdev@amazon.com>
Signed-off-by: akash kumar <akumdev@amazon.com>
Signed-off-by: akash kumar <akumdev@amazon.com>
Member
|
@akashkgit Can you mark it as a draft if it's not ready for review? |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## unstable #3544 +/- ##
============================================
+ Coverage 76.66% 76.70% +0.03%
============================================
Files 162 162
Lines 80644 80645 +1
============================================
+ Hits 61823 61856 +33
+ Misses 18821 18789 -32
🚀 New features to boost your workflow:
|
d5f9df6 to
18f6f2c
Compare
Signed-off-by: akash kumar <akumdev@amazon.com>
Signed-off-by: Akash Kumar <45854686+akashkgit@users.noreply.github.com>
Signed-off-by: akash kumar <akumdev@amazon.com>
Signed-off-by: akash kumar <akumdev@amazon.com>
Signed-off-by: akash kumar <akumdev@amazon.com>
madolson
reviewed
May 8, 2026
| #define IO_IGNITION_CPU_SYS 30.0 | ||
| #define IO_IGNITION_CPU_SYS_LOW 5.0 | ||
| #define IO_IGNITION_CPU_USER 50.0 | ||
| #define IO_IGNITION_MAIN_THREAD_ACTIVE_PERCENT 30 |
Member
There was a problem hiding this comment.
Let's still leave a comment for why this value is at 30%. We don't need to necessarily test this more comprehensively, but it would be good to at least acknowledge we haven't verified this is necessarily optimal.
Signed-off-by: akash kumar <akumdev@amazon.com>
|
Benchmark ran on this commit: RPS Benchmark Comparison: unstable vs 4b1aee5No significant changesNo statistically significant changes detected across 16 test(s). 16 with no significant change Click to expand full comparison tablesdata_size = 16
data_size = 96
Configuration:
Legend:
Statistical Notes:
|
madolson
approved these changes
May 12, 2026
madolson
reviewed
May 12, 2026
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com> Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
madolson
approved these changes
May 12, 2026
sarthakaggarwal97
pushed a commit
to sarthakaggarwal97/valkey
that referenced
this pull request
May 12, 2026
Follow-ups to the IO-threads redesign (valkey-io#3324 / valkey-io#3367). Bundles several independent fixes and refactors: - tryOffloadFreeObjToIOThreads: only offload the free of the sds buffer (which the I/O thread allocated). The robj itself is main-thread-owned and is now freed on the main thread. - trySendWriteToIOThreads: defer clearing last_header until after a successful enqueue, so the main thread can't mutate a header the I/O thread is concurrently reading. - evictClients: drop the pending_freed / close_asap bookkeeping and rely on freeClient's return value to bump stat_evictedclients. The old accounting double-counted protected clients and could under-evict. - Make MPSC/SPMC/SPSC queue sizes a runtime parameter on *Init(q, size) instead of compile-time *_QUEUE_SIZE macros in queues.h. The I/O-threads sizes (16384 / 4096 / 4096) now live as constants in io_threads.c; tests pass their own sizes. - updateIOThreads uses io_shared_outbox.queue_size instead of the macro. - Add function-level doc comments to queues.h. - Rename IOThreadFreeArgv/IOThreadPoll → ioThreadFreeArgv/ioThreadPoll for consistency. - Remove unused job_handler typedef; fix static_assert message (7 → 8). clang-format pass; Makefile tab→spaces nit. - Replace the getrusage(RUSAGE_THREAD) sys/user CPU sampling (and its #ifdef RUSAGE_THREAD fallback) with server.stat_active_time. One portable signal, one threshold (IO_IGNITION_MAIN_THREAD_ACTIVE_PERCENT = 30) replaces the dual sys>30% / sys>5%+user>50% rules. - Renames STATS_METRIC_MAIN_THREAD_CPU_{SYS,USER} → STATS_METRIC_MAIN_THREAD_ACTIVE_TIME. --------- Signed-off-by: akash kumar <akumdev@amazon.com> Signed-off-by: Akash Kumar <45854686+akashkgit@users.noreply.github.com> Signed-off-by: Madelyn Olson <madelyneolson@gmail.com> Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
lucasyonge
pushed a commit
that referenced
this pull request
May 12, 2026
Follow-ups to the IO-threads redesign (#3324 / #3367). Bundles several independent fixes and refactors: - tryOffloadFreeObjToIOThreads: only offload the free of the sds buffer (which the I/O thread allocated). The robj itself is main-thread-owned and is now freed on the main thread. - trySendWriteToIOThreads: defer clearing last_header until after a successful enqueue, so the main thread can't mutate a header the I/O thread is concurrently reading. - evictClients: drop the pending_freed / close_asap bookkeeping and rely on freeClient's return value to bump stat_evictedclients. The old accounting double-counted protected clients and could under-evict. - Make MPSC/SPMC/SPSC queue sizes a runtime parameter on *Init(q, size) instead of compile-time *_QUEUE_SIZE macros in queues.h. The I/O-threads sizes (16384 / 4096 / 4096) now live as constants in io_threads.c; tests pass their own sizes. - updateIOThreads uses io_shared_outbox.queue_size instead of the macro. - Add function-level doc comments to queues.h. - Rename IOThreadFreeArgv/IOThreadPoll → ioThreadFreeArgv/ioThreadPoll for consistency. - Remove unused job_handler typedef; fix static_assert message (7 → 8). clang-format pass; Makefile tab→spaces nit. - Replace the getrusage(RUSAGE_THREAD) sys/user CPU sampling (and its #ifdef RUSAGE_THREAD fallback) with server.stat_active_time. One portable signal, one threshold (IO_IGNITION_MAIN_THREAD_ACTIVE_PERCENT = 30) replaces the dual sys>30% / sys>5%+user>50% rules. - Renames STATS_METRIC_MAIN_THREAD_CPU_{SYS,USER} → STATS_METRIC_MAIN_THREAD_ACTIVE_TIME. --------- Signed-off-by: akash kumar <akumdev@amazon.com> Signed-off-by: Akash Kumar <45854686+akashkgit@users.noreply.github.com> Signed-off-by: Madelyn Olson <madelyneolson@gmail.com> Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
akashkgit
added a commit
to akashkgit/valkey
that referenced
this pull request
May 18, 2026
This reverts commit fdd9039. Signed-off-by: akash kumar <akumdev@amazon.com>
This was referenced May 18, 2026
madolson
pushed a commit
that referenced
this pull request
May 18, 2026
roshkhatri
added a commit
to roshkhatri/valkey
that referenced
this pull request
Jun 8, 2026
Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
madolson
pushed a commit
that referenced
this pull request
Jun 9, 2026
This regression is still present in 9.1 GA as the cherrypick of revert commit was missed during release. Re-applies #3544 (reverted in #3756 due to ~20% SET regression) with the performance fix from #3760. **Root Cause:** The original #3544 changed `tryOffloadFreeObjToIOThreads` to only offload the SDS buffer free to IO threads, freeing the `robj` shell on the main thread. I carried out profiling for the change and it showed that freeing the `robj` shell on the main thread became the prime main-thread hotspot (~10% CPU), while IO threads shifted from doing real `jemalloc` work to spinning idle on `spmcDequeue`. **Fix**: Keep `tryOffloadFreeObjToIOThreads` offloading the entire robj (`decrRefCount`) to the IO thread. Cross-thread `zfree` is safe with `jemalloc`. This PR includes all cleanup work from #3544 so - - `trySendWriteToIOThreads`: defer clearing `last_header` until after successful enqueue - `evictClients`: simplified bookkeeping - Queue sizes as runtime parameters instead of compile-time macros - IO ignition policy using `stat_active_time` instead of `getrusage` - Function renames (`IOThreadFreeArgv` --> `ioThreadFreeArgv`, etc.) and doc comments **Benchmark** on (Graviton4 c8gb.metal-48xl): Config: SET, 128B values, 9 IO threads, pipeline=10, 1600 clients - Same as Valkey official method | Version | Throughput | |---------|-----------| | Unstable + original #3544 | ~1,554K rps | | Unstable + this PR | ~2,116K rps | <details> <summary>Diff vs original #3544 (perf fix)</summary> ```diff diff --git a/src/io_threads.c b/src/io_threads.c --- a/src/io_threads.c +++ b/src/io_threads.c @@ // IO thread handler case JOB_REQ_FREE_OBJ: - zfree(data); + decrRefCount(data); break; @@ // tryOffloadFreeObjToIOThreads - /* We offload only the free of the ptr that may be allocated by the I/O thread. - * The object itself was allocated by the main thread and will be freed by the main thread. */ - void *job = tagJob(sdsAllocPtr(objectGetVal(obj)), JOB_REQ_FREE_OBJ); + void *job = tagJob(obj, JOB_REQ_FREE_OBJ); if (unlikely(spmcEnqueue(&io_shared_inbox, job) == false)) return C_ERR; - objectSetVal(obj, NULL); - decrRefCount(obj); io_jobs_submitted++; ``` </details> --------- Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
ranshid
added a commit
that referenced
this pull request
Jun 11, 2026
# Backport sweep for 9.1 Automated cherry-picks from PRs marked "To be backported". ## Applied | Source PR | Title | Detail | |---|---|---| | #3743 | Fix buffered_reply assert in HFE commands with module keyspace notifications | cherry-picked in a prior sweep | | #3766 | Fix flaky block_keyspace_notification test for HGETDEL notify race | cherry-picked in a prior sweep | | #3800 | Fix heap-use-after-free in ACL LOAD when client free is deferred | cherry-picked in a prior sweep | | #3723 | Fix double-finish and RESP reply violation in cluster slot migration | cherry-picked in a prior sweep | | #3872 | Redacting customer information when hide_user_data_from_log is true in rdb.c, networking.c, debug.c and t_hash | cherry-picked in a prior sweep | | #3846 | Fix use-after-free in VM_RegisterClusterMessageReceiver | cherry-picked in a prior sweep | | #3806 | Add ALL_DBS flag to CLUSTER FLUSHSLOT for database-level ACL | cherry-picked in a prior sweep | | #3847 | Harden SENTINEL commands and config rewrite against control-character injection | | | #3801 | Validate every DB clause in COPY against ACL db= permissions | | | #3851 | Replace AUTOMATION_PAT with Valkeyrie Bot GitHub App token | | | #3848 | Fix cluster AUX-field control-character and delimiter injection | | | #3544 | Revert "IO-Threads redesign cleanup work (#3544)" | cherry-picked in a prior sweep | | #3888 | Report exact dbid for COPY in ACL LOG when db= access is denied | conflicts resolved by Claude Code | | #3942 | Fix shard_id format specifier in UPDATE message log | | | #3941 | Avoid random() % 0 undefined behaviour when cluster-node-timeout < 30 | | --- *Generated by valkey-ci-agent using Claude Code.* --------- Signed-off-by: Binbin <binloveplay1314@qq.com> Signed-off-by: Ran Shidlansik <ranshid@amazon.com> Signed-off-by: chx9 <lovelypiska@outlook.com> Signed-off-by: zackcam <zackcam@amazon.com> Signed-off-by: Eran Ifrah <eifrah@amazon.com> Signed-off-by: Jules Lasarte <lasartej@amazon.com> Signed-off-by: jjuleslasarte <jules.lasarte@gmail.com> Signed-off-by: akash kumar <akumdev@amazon.com> Co-authored-by: Binbin <binloveplay1314@qq.com> Co-authored-by: Ran Shidlansik <ranshid@amazon.com> Co-authored-by: lovelypiska <lovelypiska@outlook.com> Co-authored-by: zackcam <zackcam@amazon.com> Co-authored-by: eifrah-aws <eifrah@amazon.com> Co-authored-by: jjuleslasarte <jules.lasarte@gmail.com> Co-authored-by: Jules Lasarte <lasartej@amazon.com> Co-authored-by: Akash Kumar <45854686+akashkgit@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-ups to the IO-threads redesign (#3324 / #3367). Bundles several independent fixes and refactors:
clang-format pass; Makefile tab→spaces nit.