Skip to content

Fix hang when using --reconnect-interval with --rate-limiting#348

Merged
fcostaoliveira merged 1 commit intomasterfrom
reconnect.fix
Feb 26, 2026
Merged

Fix hang when using --reconnect-interval with --rate-limiting#348
fcostaoliveira merged 1 commit intomasterfrom
reconnect.fix

Conversation

@fcostaoliveira
Copy link
Collaborator

@fcostaoliveira fcostaoliveira commented Feb 26, 2026

Summary

  • Fix memtier_benchmark hanging indefinitely when --reconnect-interval and --rate-limiting are used together (regression introduced in 2.2.1 via Issue #285, Fix rate-limiting with cluster-mode #286)
  • The rate-limiting timer was only created on the first connection. After --reconnect-interval triggered a disconnect/reconnect cycle, disconnect() properly freed the timer but handle_event() never recreated it, leaving m_request_per_cur_interval permanently at 0 and causing fill_pipeline() to return without sending any requests
  • Moved timer creation outside the first-connection guard so it is recreated on every successful connect/reconnect when m_event_timer is NULL
  • Added test_reconnect_interval_with_rate_limiting integration test matching the exact failing scenario (--reconnect-interval=1 --rate-limiting=1)

Test plan

  • New test test_reconnect_interval_with_rate_limiting passes (would hang forever without fix)
  • Existing test_short_reconnect_interval still passes (no regression)
  • CI passes

🤖 Generated with Claude Code


Note

Medium Risk
Touches core connection/event-loop behavior and rate limiting, so regressions could affect request pacing or reconnect stability. Change is small and covered by a new integration test that previously would hang.

Overview
Prevents memtier_benchmark from stalling after a reconnect when rate limiting is enabled by recreating the request-rate timer on successful (re)connect whenever m_event_timer is NULL (instead of only on the first connection).

Adds test_reconnect_interval_with_rate_limiting to exercise --reconnect-interval=1 --rate-limiting=1 and assert the run completes and produces non-zero ops in mb.json.

Written by Cursor Bugbot for commit d4e3296. This will update automatically on new commits. Configure here.

The rate-limiting timer was only created on the first connection
(when get_reqs_processed() == 0). After a reconnect triggered by
--reconnect-interval, disconnect() properly freed the timer, but
handle_event() never recreated it because requests had already been
processed. This left m_request_per_cur_interval permanently at 0,
causing fill_pipeline() to return immediately on every call.

Move timer creation outside the first-connection guard so it is
recreated on every successful connect/reconnect when m_event_timer
is NULL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@fcostaoliveira
Copy link
Collaborator Author

cursor review

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

@jit-ci
Copy link

jit-ci bot commented Feb 26, 2026

❌ Jit Scanner failed - Our team is investigating

Jit Scanner failed - Our team has been notified and is working to resolve the issue. Please contact support if you have any questions.


💡 Need to bypass this check? Comment @sera bypass to override.

@jit-ci
Copy link

jit-ci bot commented Feb 26, 2026

🛡️ Jit Security Scan Results

CRITICAL HIGH MEDIUM

✅ No security findings were detected in this PR


Security scan by Jit

@fcostaoliveira fcostaoliveira merged commit 47a1e02 into master Feb 26, 2026
76 of 78 checks passed
filipecosta90 pushed a commit that referenced this pull request Feb 26, 2026
The rate-limiting timer was only created on the first connection
(when get_reqs_processed() == 0). After a reconnect triggered by
--reconnect-interval, disconnect() properly freed the timer, but
handle_event() never recreated it because requests had already been
processed. This left m_request_per_cur_interval permanently at 0,
causing fill_pipeline() to return immediately on every call.

Move timer creation outside the first-connection guard so it is
recreated on every successful connect/reconnect when m_event_timer
is NULL.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@fcostaoliveira fcostaoliveira mentioned this pull request Feb 26, 2026
2 tasks
fcostaoliveira added a commit that referenced this pull request Feb 26, 2026
* Add clang-format for code style enforcement (#336)

Cherry-picked and re-applied clang-format configuration, CI workflow,
Makefile targets, and DEVELOPMENT.md docs from master. Source files
reformatted against the 2.2.x branch codebase.

* Concurrent ubuntu test jobs for faster CI (#337)

* Concurrent ubuntu test jobs for faster CI

* Prevent duplicate workflow runs on push+PR by filtering branches

* Fixed coverage workflow

* Concurrent ASAN, TSAN, and UBSAN test jobs using matrix strategy

* Updated org from redislabs to redis (#339)

* Fix rate-limit induced hanging at test completion (#340)

* Fix rate-limit induced hanging at test completion

* include a null check for conns in all_connections_idle

* Remove crufty ctx reference

* Extend CI with higher shard count scenario.

* Extra logging on 99 shards scenario

* Verbose was already defined on CI. Using 49 shards to expedite CI

* Using RLTEST_DEBUG to avoid overriding old behaviour

* Shard count 99 in rate-limiting test

---------

Co-authored-by: fcostaoliveira <filipe@redis.com>

* configure: Respect user-supplied CXXFLAGS (#342)

* Add AGENTS.md and CLAUDE.md for AI assistant guidelines (#343)

Add documentation to help AI assistants work effectively with the
memtier_benchmark codebase, following the https://agents.md/ conventions.

AGENTS.md includes:
- Project overview and repository structure
- Build system and commands (autotools)
- Code style (clang-format)
- Testing with RLTest (standalone, cluster, TLS, sanitizers)
- Key technical details
- Common development tasks
- Debugging guide (GDB, crash handler, core dumps, sanitizers)
- License header requirements

CLAUDE.md points to AGENTS.md for shared guidelines.

References:
- https://agents.md/ - Standard for AI agent documentation
- https://docs.anthropic.com/en/docs/agents - Anthropic agent guidelines

* Use latest rltest and set cluster-start-timeout to accomodate large shard count (#345)

* CI: trigger workflows on semver release branches

Add branch pattern '[0-9]+.[0-9]+' to push/pull_request triggers for
ci, code-style, asan, tsan, and ubsan workflows so CI runs on PRs
targeting release branches like 2.2.

* Fix hang when using --reconnect-interval with --rate-limiting (#348)

The rate-limiting timer was only created on the first connection
(when get_reqs_processed() == 0). After a reconnect triggered by
--reconnect-interval, disconnect() properly freed the timer, but
handle_event() never recreated it because requests had already been
processed. This left m_request_per_cur_interval permanently at 0,
causing fill_pipeline() to return immediately on every call.

Move timer creation outside the first-connection guard so it is
recreated on every successful connect/reconnect when m_event_timer
is NULL.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Bumping version to 2.2.2

---------

Co-authored-by: Tristan Schneiter <tschneiter@figma.com>
Co-authored-by: LINKIWI <LINKIWI@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants