Skip to content

Optimize SET key value EX/PX/EXAT ttl to reduce calls of rewriteClientCommandVector#3279

Merged
enjoy-binbin merged 1 commit into
valkey-io:unstablefrom
enjoy-binbin:set_flag
Mar 11, 2026
Merged

Optimize SET key value EX/PX/EXAT ttl to reduce calls of rewriteClientCommandVector#3279
enjoy-binbin merged 1 commit into
valkey-io:unstablefrom
enjoy-binbin:set_flag

Conversation

@enjoy-binbin

@enjoy-binbin enjoy-binbin commented Feb 28, 2026

Copy link
Copy Markdown
Member

If the command is in the form of "SET key value EX/PX/EXAT ttl",
then we don't need to rewrite the entire command vector.

In addition to saving the rewriting of the command vector, we also
save one command table lookup in this case ince we don't need to
lookup SET command again.

We already have many relevant test coverages in expire.tcl. This
PR does not change anything around the propagate.

Like 2544755.

…tCommandVector

If the command is in the form of "SET key value EX/PX/EXAT ttl",
then we don't need to rewrite the entire command vector.

In addition to saving the rewriting of the command vector, we also
save one command table lookup in this case ince we don't need to
lookup SET command again.

Like 2544755.

Signed-off-by: Binbin <binloveplay1314@qq.com>
@zuiderkwast

Copy link
Copy Markdown
Contributor

In 53d1acd we decided to always replica in absolute timestamps. I don't remember the context, but guess matters when the replication is lagging and in AOF files.

@enjoy-binbin

enjoy-binbin commented Mar 10, 2026

Copy link
Copy Markdown
Member Author

In 53d1acd we decided to always replica in absolute timestamps

This PR did not change it, we still rewrite it as absolute timestamps. (And btw, we have a lot of abs-ttl test around it in expire.tcl, so we did not change anything)

The different is:

Before this PR, a SET key value EX/PX/EXAT ttl will be rewrite to SET key value PXAT abs_ttl, we call rewriteClientCommandVector and rewrite the whole argv vector.

And in this PR, a SET key value EX/PX/EXAT ttl is still will be rewrite to SET key value PXAT abs_ttl, but we only call rewriteClientCommandArgument twice to rewrite the two arg. Like rewrite EX to PXAT, and rewrite ttl to abs_ttl

If a command is SET key value EX/PX/EXAT ttl

rewriteClientCommandVector rewrite the whole argv:

  • set to set
  • key to key
  • value to value
  • EX/PX/EXAT to PXAT
  • ttl to abs-ttl

And in this PR, we only rewrite those args that need rewriting:

- EX/PX/EXAT to PXAT
- ttl to abs-ttl

@zuiderkwast zuiderkwast left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still rewrite it as absolute timestamps

Sorry, I didn't read the code properly. Looks good!

@sarthakaggarwal97 sarthakaggarwal97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! @enjoy-binbin would it make sense to add a test that verifies this argc == 5 path?

@enjoy-binbin

Copy link
Copy Markdown
Member Author

would it make sense to add a test that verifies this argc == 5 path?

we already had a lot of it

    # Start a new server with empty data and AOF file.
    start_server {overrides {appendonly {yes} appendfsync always} tags {external:skip}} {
        test {All time-to-live(TTL) in commands are propagated as absolute timestamp in milliseconds in AOF} {
            # This test makes sure that expire times are propagated as absolute
            # times to the AOF file and not as relative time, so that when the AOF
            # is reloaded the TTLs are not being shifted forward to the future.
            # We want the time to logically pass when the server is restarted!

            set aof [get_last_incr_aof_path r]

            # Apply each TTL-related command to a unique key
            # SET commands
            r set foo1 bar ex 100
            r set foo2 bar px 100000
            r set foo3 bar exat [expr [clock seconds]+100]
            r set foo4 bar PXAT [expr [clock milliseconds]+100000]

@enjoy-binbin enjoy-binbin merged commit a7d919a into valkey-io:unstable Mar 11, 2026
57 checks passed
@enjoy-binbin enjoy-binbin deleted the set_flag branch March 11, 2026 02:35
@codecov

codecov Bot commented Mar 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.00%. Comparing base (834dc83) to head (022aff9).
⚠️ Report is 34 commits behind head on unstable.

Additional details and impacted files
@@       Coverage Diff        @@
##   unstable   #3279   +/-   ##
================================
================================
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

JimB123 pushed a commit that referenced this pull request Mar 19, 2026
…tCommandVector (#3279)

If the command is in the form of "SET key value EX/PX/EXAT ttl",
then we don't need to rewrite the entire command vector.

In addition to saving the rewriting of the command vector, we also
save one command table lookup in this case ince we don't need to
lookup SET command again.

We already have many relevant test coverages in expire.tcl. This
PR does not change anything around the propagate.

Somehow like 2544755.

Signed-off-by: Binbin <binloveplay1314@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants