fix: don't rate limit jobs when touching#3699
Merged
johanneskoester merged 1 commit intosnakemake:mainfrom Aug 18, 2025
Merged
Conversation
Contributor
📝 WalkthroughWalkthroughAdjusted JobRateLimiter initialization in JobScheduler.init: the limiter is now created only when not in dry-run, not in touch mode, and max_jobs_per_timespan is set. This disables rate limiting during touch mode. Scheduling logic remains unchanged aside from bypassing rate limiting when no limiter is present. Changes
Sequence Diagram(s)sequenceDiagram
actor User
participant Scheduler
participant RateLimiter
User->>Scheduler: Initialize JobScheduler
alt not dry-run AND not touch AND max_jobs_per_timespan
Scheduler->>RateLimiter: Create limiter
else
Scheduler->>Scheduler: limiter = None
end
User->>Scheduler: Schedule jobs
alt limiter exists
Scheduler->>RateLimiter: Check/Enforce rate limit
RateLimiter-->>Scheduler: Allowed/Delayed
else
Scheduler->>Scheduler: Proceed without rate limiting
end
Estimated code review effort🎯 2 (Simple) | ⏱️ ~7 minutes Suggested reviewers
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. 📜 Recent review detailsConfiguration used: CodeRabbit UI 💡 Knowledge Base configuration:
You can enable these sources in your CodeRabbit configuration. 📒 Files selected for processing (1)
🧰 Additional context used📓 Path-based instructions (1)**/*.py⚙️ CodeRabbit Configuration File
Files:
🧬 Code Graph Analysis (1)src/snakemake/scheduler.py (2)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (56)
🔇 Additional comments (1)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
johanneskoester
approved these changes
Aug 18, 2025
johanneskoester
pushed a commit
that referenced
this pull request
Aug 29, 2025
🤖 I have created a release *beep* *boop* --- ## [9.10.0](v9.9.0...v9.10.0) (2025-08-19) ### Features * migrate to scheduler plugin interface and scheduler plugins ([#3676](#3676)) ([26fcd38](26fcd38)) ### Bug Fixes * don't rate limit jobs when touching ([#3699](#3699)) ([9c499e5](9c499e5)) * raise an error when different rules produce identical (temp) output ([#3667](#3667)) ([f627176](f627176)) * silence messages on module load ([#3688](#3688)) ([b13e9c8](b13e9c8)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
kjohnsen
pushed a commit
to kjohnsen/snakemake
that referenced
this pull request
Dec 15, 2025
Came across this while `--touch`ing many files, but it seems having something like `max-jobs-per-timespan: "100/1m"` in a profile will also limit the rate for files getting touched (which is a very different operation to actually submitting jobs). Previously you would get many messages for > Job rate limit reached, waiting for free slots. Whereas now you should only encounter that message if you are not in dryrun or touch mode. ### QC <!-- Make sure that you can tick the boxes below. --> * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Touch mode now bypasses job submission rate limiting, enabling faster execution when only updating timestamps. * Users with a configured submission cap will no longer experience artificial delays in touch runs; normal runs continue to honor rate limits, and dry-run behavior is unchanged. * Scheduling order and resource checks remain the same; only the rate limiting is skipped in touch mode, improving responsiveness without altering other execution semantics. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
kjohnsen
pushed a commit
to kjohnsen/snakemake
that referenced
this pull request
Dec 15, 2025
🤖 I have created a release *beep* *boop* --- ## [9.10.0](snakemake/snakemake@v9.9.0...v9.10.0) (2025-08-19) ### Features * migrate to scheduler plugin interface and scheduler plugins ([snakemake#3676](snakemake#3676)) ([26fcd38](snakemake@26fcd38)) ### Bug Fixes * don't rate limit jobs when touching ([snakemake#3699](snakemake#3699)) ([9c499e5](snakemake@9c499e5)) * raise an error when different rules produce identical (temp) output ([snakemake#3667](snakemake#3667)) ([f627176](snakemake@f627176)) * silence messages on module load ([snakemake#3688](snakemake#3688)) ([b13e9c8](snakemake@b13e9c8)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Came across this while
--touching many files, but it seems having something likemax-jobs-per-timespan: "100/1m"in a profile will also limit the rate for files getting touched (which is a very different operation to actually submitting jobs).Previously you would get many messages for
Whereas now you should only encounter that message if you are not in dryrun or touch mode.
QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).Summary by CodeRabbit