Skip to content

fix: restore accidentally changed behavior of --cores. If not specified, Snakemake now complains again, asking for --cores to be specified (of --jobs in case of remote exec). As before, if a default is desired, it can be easily set via a profile#3531

Merged
johanneskoester merged 2 commits intomainfrom
fix/restore-cores-behavior
Apr 14, 2025

Conversation

@johanneskoester
Copy link
Copy Markdown
Contributor

@johanneskoester johanneskoester commented Apr 14, 2025

Description

QC

  • The PR contains a test case for the changes or the changes are already covered by an existing test case.
  • The documentation (docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).

Summary by CodeRabbit

  • Refactor
    • Updated the logic for assigning CPU cores, removing the automatic fallback to available CPU count when not explicitly set.
  • Bug Fixes
    • Improved error handling to avoid raising null exceptions after failed executions.
  • Tests
    • Modified test commands to explicitly specify single-core usage during runs.

…ed, Snakemake now complains again, asking for --cores to be specified (of --jobs in case of remote exec). As before, if a default is desired, it can be easily set via a profile
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 14, 2025

📝 Walkthrough
## Walkthrough

The change removes a fallback assignment in the command-line interface logic where the number of CPU cores (`args.cores`) would default to the system's available CPU count if not explicitly set. After this modification, if no value is provided for `args.cores` through prior logic, it will remain `None`. Additionally, test code was updated to avoid raising `None` exceptions and to explicitly specify `--cores 1` in certain test Snakemake command invocations. No other logic or function signatures are affected.

## Changes

| File(s)                      | Change Summary                                                                                 |
|------------------------------|-----------------------------------------------------------------------------------------------|
| src/snakemake/cli.py         | Removed fallback code that set `args.cores` to the available CPU count if it was still `None`. |
| tests/common.py              | Modified error handling to raise exceptions only if they are not `None` after failed runs.     |
| tests/tests.py               | Added `--cores 1` option to Snakemake shell commands in two test functions.                    |

## Sequence Diagram(s)

```mermaid
sequenceDiagram
    participant User
    participant CLI
    User->>CLI: Provide command-line arguments (may omit --cores)
    CLI->>CLI: Process arguments
    alt --cores provided or set by earlier logic
        CLI->>CLI: Use specified value for args.cores
    else --cores not set
        CLI->>CLI: args.cores remains None (no fallback assignment)
    end

</details>

<!-- walkthrough_end -->


---

<details>
<summary>📜 Recent review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between 10e544a86067dee20abd2e7cc62a59d7a208c98e and 5531ee06be5a1df2d763cabafcadc2b650deb19f.

</details>

<details>
<summary>📒 Files selected for processing (2)</summary>

* `tests/common.py` (1 hunks)
* `tests/tests.py` (2 hunks)

</details>

<details>
<summary>✅ Files skipped from review due to trivial changes (1)</summary>

* tests/tests.py

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>📓 Path-based instructions (1)</summary>

<details>
<summary>`**/*.py`: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the `s...</summary>

> `**/*.py`: Do not try to improve formatting.
> Do not suggest type annotations for functions that are defined inside of functions or methods.
> Do not suggest type annotation of the `self` argument of methods.
> Do not suggest type annotation of the `cls` argument of classmethods.
> Do not suggest return type annotation if a function or method does not contain a `return` statement.

- `tests/common.py`

</details>

</details>

</details>

<details>
<summary>⏰ Context from checks skipped due to timeout of 90000ms (34)</summary>

* GitHub Check: tests (10, windows-latest, py312)
* GitHub Check: tests (10, ubuntu-latest, py312)
* GitHub Check: tests (10, ubuntu-latest, py311)
* GitHub Check: tests (9, windows-latest, py312)
* GitHub Check: tests (9, ubuntu-latest, py312)
* GitHub Check: tests (9, ubuntu-latest, py311)
* GitHub Check: tests (8, windows-latest, py312)
* GitHub Check: tests (8, ubuntu-latest, py312)
* GitHub Check: tests (8, ubuntu-latest, py311)
* GitHub Check: tests (7, windows-latest, py312)
* GitHub Check: tests (7, ubuntu-latest, py312)
* GitHub Check: tests (7, ubuntu-latest, py311)
* GitHub Check: tests (6, windows-latest, py312)
* GitHub Check: tests (6, ubuntu-latest, py312)
* GitHub Check: tests (6, ubuntu-latest, py311)
* GitHub Check: tests (5, windows-latest, py312)
* GitHub Check: tests (5, ubuntu-latest, py312)
* GitHub Check: tests (5, ubuntu-latest, py311)
* GitHub Check: tests (4, windows-latest, py312)
* GitHub Check: tests (4, ubuntu-latest, py312)
* GitHub Check: tests (4, ubuntu-latest, py311)
* GitHub Check: tests (3, macos-latest, py312)
* GitHub Check: tests (3, windows-latest, py312)
* GitHub Check: tests (3, ubuntu-latest, py312)
* GitHub Check: tests (3, ubuntu-latest, py311)
* GitHub Check: tests (2, macos-latest, py312)
* GitHub Check: tests (2, windows-latest, py312)
* GitHub Check: tests (2, ubuntu-latest, py312)
* GitHub Check: tests (2, ubuntu-latest, py311)
* GitHub Check: tests (1, macos-latest, py312)
* GitHub Check: tests (1, windows-latest, py312)
* GitHub Check: tests (1, ubuntu-latest, py312)
* GitHub Check: tests (1, ubuntu-latest, py311)
* GitHub Check: apidocs

</details>

<details>
<summary>🔇 Additional comments (1)</summary><blockquote>

<details>
<summary>tests/common.py (1)</summary>

`451-452`: **Fix error handling to prevent raising None exception**

The conditional check added here prevents attempts to raise a `None` exception, which would result in a `TypeError: exceptions must derive from BaseException`. This is a necessary fix that aligns with the PR's goal of restoring the `--cores` behavior, as the tests would need to handle the case where no default core count is assigned.

</details>

</blockquote></details>

</details>
<!-- internal state start -->


<!-- DwQgtGAEAqAWCWBnSTIEMB26CuAXA9mAOYCmGJATmriQCaQDG+Ats2bgFyQAOFk+AIwBWJBrngA3EsgEBPRvlqU0AgfFwA6NPEgQAfACgjoCEYDEZyAAUASpETZWaCrKNwSPbABsvkCiQBHbGlcSHFcLzpIACIAM3gADy5/RAJ/dAYGeCUMXDQfeQZYTFJ6ARJiiXh8PnxY3TAmFOj0WloUxGl0LCQHDwB3WEoPXCHIcsrq2vrRjwADCCbpOf5ucXwerABlDDQAaxJmfYG0ZGwMeFz2at2C9C8af1oNGDGiko8UtK7Znn8q/DYGQVNAAviDYYAGhQMzGC0aNWWq3WPWQGHwoUQ3FE8HidGhO2ORwOkH68B8kHR/XQRG0WF4LDWYTGQMoYXw9mxWVi8nhS0QKwAFDVIPChIIBSgsL8GKcPHU/IcMR4SAlRHgbgBKF5wVBkBwpZnUSCsijIZwqhLcLzwLIReRofoW/iwz7SQEUBgefJefCylGKoLwfxsXJnDCRRDmyBKWJobyhCT5YIoZCdRPwNDoP51ckkHVjWxShhebBKaM0VKMOXshRSPgy4oYUjmjD0JgYeIUZjIUbG2h+xzsag3E3cWjULr9YYKDCIbLDWjQ/KjQFEWCUjnMRS420jja9jm/LwlbBoUj8Pixc5iG75dTwKcz8heqPOWQFjzcby+fxBEKkqc9jYAIzDqDQZTyL8pqQOKTbkIgez4CEbKYPQqALrku5REBGCOOUfAAMwAKyEQAjFKzIeISBzEm63D4POaQfq8HjXhSSh5OSrb0LQSAMEC84bNWWDlJAsSAm26ChLAuC4NwiAcAA9EpRDqLAIEaEwzBKYguy0ccun6Ychnfj4SkkeRGhGPoxjgFAZD0Aq8YEMQZDKBBCisOwXC8PwwiiOIUgyIUijKKo6haDotkmFAurIKgmA4K5pDkFQnnaaGnB+I6wFOC44yhUoVARZo2gNIYdmmAYiCekZRKGSW8AaNwsgcAY0SdQYFiQAAggAkm5aWTvQDj5fICrvM20huGMIb4FI7ZheMvoMHsRqYiQoRzM4RCIFpiKSgQVGUvhbLOUm5IqJEkAAMJWAAqgohq4igoROsgcwAHIbCQKzFMCZAxiQkQQS8Vj/NUQJeLI0KvTtFB7QdKQrB99jiBS32/SsaCxI89gsB4HZ8Si+ToFG8BEBgWWIHD72Al4ZTehTVNRL8Salm6uDYBQ5CQaKoLaCeAiRAA+gw37i5JuCCpqcwvD9/TQqMqBxj4AhoGt5PzlTWWpvwYFyfiBMC4j+38isIZ0p9P3kCsr3optoRyJAJDODabK+mpDCfoqVYKr814YLewlobOuAUPgvixL61JW5cJrB02pRw8HpZ8c2kBNrQNqZwqqrqmkPClmpWDpuIzY8Tm8SRHwqTONz3DWUY3WWL1DweTch4nUoJbOPuc4uq7Vo1J5IrfiLtqu1h4gzQYUA/Ywyc/EeYw67s3OGiKvcnulXdD6qDEUGPfATzaDDT+Ej7INO6RHEolEq8gfGxLE1mddENnVZWuCIEpmUbBam1DqXUeoDSGh5KIY0jgFUmsvRAs0ibL0gNuF+18TqUEjnwbOuciCQC9lPXG+MsxxjzPQGiJkSQFwEgGBOvw5gUHOCsIOIcMDg0hoCRAMM4aumHoXUcaNzgOEyNIRA7Fuj0CSqqL0awBFAVlGsHmxsZTLX6AzegN4NgkzvHcfwYAqBIBGGMaRJBZGANYmOCcGVlpoDaNGYmD4NhkyKKIdax0DGdAwQkGRAYNgwxhG9fWjtRS2z+gWVAvASBSDDDlJAlw8FZixnbYePjRyCBEGISispOg3yGOkOYwjXySlQHMAAYvkToKwBB4E3Ck0xAY0anHXnQX2Xwh4qNyJHaOscJHa0oCiZA8csA3mXs8Furc+odz3geWsvwd79wGQfEex8ojjxAufS+D456hJAZ/ee38Qh/x/vtVq7UP4TPAalSBo1HAwImvUKaLY3BqLCABFhiyE5zGOUpY5QDUYzlQdhegx1bHArhIsQ6kAyIrHwGY6Uq8SDBlFIgIYPgGDMFoDjRGQ4YkQk2BIfAex4mQAoXRF4WwuS7llAUVOoof6i0wFkdg4sbSyzmMrNeqLfCZTDo8qIsRI7MFFNEPSDUSQQrnAuCgYBGWPlyH1AAvAABkgAAIQVQkaIKxjpzBFcZOiDQOzzmKjK4OcrQi9WVWqjVhrIVkS1eS+AYFd7cMol8kIotejBEIoRAAbGRUW3Amlso5R4FFINuUsCOFJPl9ABUsGFaKgy4qwCIFoEKtA3A1h0koFq2suqk2UI8BANNGas1cTSraw09r5asU8Xy5Ah9z7qH8ViHEPITp4WYARIe/JayshjDzYlvxSXHD8EIx+QxPG/MgIvDEeT8H4G9tCTBIocHxOhCKDsEco7iR6Y0qZLTxlgKmQPbuczRC7zPUso+J9PCTwvtcWeCD56zo5A22ZYx5nTMHiKde1AlHIHzssu9Z8p5PvQbfDw998yQAAPIRnkJcR4txXlVneaOalXgb4Ap3HiegZJRiWJGvYLlXlo2jQjvE/auyv6VVitPJy9QXKECuelKImUfI5WpNA98hUFDFRUGoMq0UwCGAY5ldQnraCIFFpDEg05aCi3rsfCqVUoBkSVSQYiAAWHTaAAAcvqlW+oAOxKBIAAJiVSoWglmSCmcyL6yzaBiIAE5aCmbQNZgzDA3MGeLWJgwDHiKkTIiQEgJnyjETQGRWgsRLOed9YRWUGtYiyloAwSzAhfXESVUoAQZE3P1FssF+y5GpPZFk/JxTotHIVSAA= -->

<!-- internal state end -->
<!-- finishing_touch_checkbox_start -->

<details>
<summary>✨ Finishing Touches</summary>

- [ ] <!-- {"checkboxId": "7962f53c-55bc-4827-bfbf-6a18da830691"} --> 📝 Generate Docstrings

</details>

<!-- finishing_touch_checkbox_end -->
<!-- tips_start -->

---



<details>
<summary>🪧 Tips</summary>

### Chat

There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai?utm_source=oss&utm_medium=github&utm_campaign=snakemake/snakemake&utm_content=3531):

- Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  - `I pushed a fix in commit <commit_id>, please review it.`
  - `Generate unit testing code for this file.`
  - `Open a follow-up GitHub issue for this discussion.`
- Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples:
  - `@coderabbitai generate unit testing code for this file.`
  -	`@coderabbitai modularize this function.`
- PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  - `@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.`
  - `@coderabbitai read src/utils.ts and generate unit testing code.`
  - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.`
  - `@coderabbitai help me debug CodeRabbit configuration file.`

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

### CodeRabbit Commands (Invoked using PR comments)

- `@coderabbitai pause` to pause the reviews on a PR.
- `@coderabbitai resume` to resume the paused reviews.
- `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
- `@coderabbitai full review` to do a full review from scratch and review all the files again.
- `@coderabbitai summary` to regenerate the summary of the PR.
- `@coderabbitai generate docstrings` to [generate docstrings](https://docs.coderabbit.ai/finishing-touches/docstrings) for this PR.
- `@coderabbitai resolve` resolve all the CodeRabbit review comments.
- `@coderabbitai plan` to trigger planning for file edits and PR creation.
- `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository.
- `@coderabbitai help` to get help.

### Other keywords and placeholders

- Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed.
- Add `@coderabbitai summary` to generate the high-level summary at a specific location in the PR description.
- Add `@coderabbitai` anywhere in the PR title to generate the title automatically.

### CodeRabbit Configuration File (`.coderabbit.yaml`)

- You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository.
- Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information.
- If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json`

### Documentation and Community

- Visit our [Documentation](https://docs.coderabbit.ai) for detailed information on how to use CodeRabbit.
- Join our [Discord Community](http://discord.gg/coderabbit) to get help, request features, and share feedback.
- Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.

</details>

<!-- tips_end -->

@johanneskoester johanneskoester merged commit 11b2b30 into main Apr 14, 2025
49 checks passed
@johanneskoester johanneskoester deleted the fix/restore-cores-behavior branch April 14, 2025 21:04
johanneskoester pushed a commit that referenced this pull request Apr 14, 2025
🤖 I have created a release *beep* *boop*
---


##
[9.2.0](v9.1.11...v9.2.0)
(2025-04-14)


### Features

* auto detect string resources in resource definitions given by
--default-resources and --set-resources in the string yields a syntax
error when being interpreted as a python expression; this will obviate
the need to quote such expressions in many cases; further, fix
interpretation of quoted resources by removing quotes after
intepretation
([#3532](#3532))
([81f09bd](81f09bd))


### Bug Fixes

* restore accidentally changed behavior of --cores. If not specified,
Snakemake now complains again, asking for --cores to be specified (of
--jobs in case of remote exec). As before, if a default is desired, it
can be easily set via a profile
([#3531](#3531))
([11b2b30](11b2b30))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
@peterjc
Copy link
Copy Markdown
Contributor

peterjc commented Apr 16, 2025

Just got caught out by this change with the (new to me) error:

ERROR snakemake.logging:exceptions.py:181 Error: cores have to be specified for local execution (use --cores N with N being a number >= 1 or 'all')

Easy to work arround with an explicit --cores all, and the change does seem reasonable. I am commenting here mainly to get the error message in your issue tracker to help anyone else.

kjohnsen pushed a commit to kjohnsen/snakemake that referenced this pull request Dec 15, 2025
…ed, Snakemake now complains again, asking for --cores to be specified (of --jobs in case of remote exec). As before, if a default is desired, it can be easily set via a profile (snakemake#3531)

### Description

<!--Add a description of your PR here-->

### QC
<!-- Make sure that you can tick the boxes below. -->

* [x] The PR contains a test case for the changes or the changes are
already covered by an existing test case.
* [x] The documentation (`docs/`) is updated to reflect the changes or
this is not necessary (e.g. if the change does neither modify the
language nor the behavior or functionalities of Snakemake).


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Refactor**
- Updated the logic for assigning CPU cores, removing the automatic
fallback to available CPU count when not explicitly set.
- **Bug Fixes**
- Improved error handling to avoid raising null exceptions after failed
executions.
- **Tests**
- Modified test commands to explicitly specify single-core usage during
runs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
kjohnsen pushed a commit to kjohnsen/snakemake that referenced this pull request Dec 15, 2025
🤖 I have created a release *beep* *boop*
---


##
[9.2.0](snakemake/snakemake@v9.1.11...v9.2.0)
(2025-04-14)


### Features

* auto detect string resources in resource definitions given by
--default-resources and --set-resources in the string yields a syntax
error when being interpreted as a python expression; this will obviate
the need to quote such expressions in many cases; further, fix
interpretation of quoted resources by removing quotes after
intepretation
([snakemake#3532](snakemake#3532))
([81f09bd](snakemake@81f09bd))


### Bug Fixes

* restore accidentally changed behavior of --cores. If not specified,
Snakemake now complains again, asking for --cores to be specified (of
--jobs in case of remote exec). As before, if a default is desired, it
can be easily set via a profile
([snakemake#3531](snakemake#3531))
([11b2b30](snakemake@11b2b30))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants