Skip to content

fix: Adapt for dataclasses._MISSING_TYPE replaced with sentinel in Python 3.15#4211

Merged
johanneskoester merged 1 commit into
snakemake:mainfrom
musicinmybrain:py315-sentinel
Jun 1, 2026
Merged

fix: Adapt for dataclasses._MISSING_TYPE replaced with sentinel in Python 3.15#4211
johanneskoester merged 1 commit into
snakemake:mainfrom
musicinmybrain:py315-sentinel

Conversation

@musicinmybrain

@musicinmybrain musicinmybrain commented Jun 1, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Contributor

See python/cpython#149086 and PEP 661 for context.

This fixes a traceback from snakemake --help in Python 3.15.0b1:

$ git clone https://github.com/snakemake/snakemake.git
$ cd snakemake
$ uv venv --python 3.15
$ . .venv/bin/activate
(snakemake) $ uv pip install -e .
(snakemake) $ snakemake --help
Traceback (most recent call last):

  File "/usr/lib64/python3.15/argparse.py", line 2878, in print_help
    help_text = self.format_help(formatter=formatter)

TypeError: ArgumentParser.format_help() got an unexpected keyword argument 'formatter'


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/home/ben/src/forks/snakemake/src/snakemake/cli.py", line 2391, in main
    parser, args = parse_args(argv)
                   ~~~~~~~~~~^^^^^^

  File "/home/ben/src/forks/snakemake/src/snakemake/cli.py", line 1904, in parse_args
    args = parser.parse_args(argv)

  File "/home/ben/src/forks/snakemake/.venv/lib64/python3.15/site-packages/configargparse.py", line 1048, in parse_args
    args, argv = self.parse_known_args(
                 ~~~~~~~~~~~~~~~~~~~~~^
        args=args,
        ^^^^^^^^^^
    ...<3 lines>...
        ignore_help_args=False,
        ^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^

  File "/home/ben/src/forks/snakemake/.venv/lib64/python3.15/site-packages/configargparse.py", line 1266, in parse_known_args
    namespace, unknown_args = argparse.ArgumentParser.parse_known_args(
                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, args=args, namespace=namespace
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^

  File "/usr/lib64/python3.15/argparse.py", line 2126, in parse_known_args
    return self._parse_known_args2(args, namespace, intermixed=False)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/lib64/python3.15/argparse.py", line 2155, in _parse_known_args2
    namespace, args = self._parse_known_args(args, namespace, intermixed)
                      ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/lib64/python3.15/argparse.py", line 2406, in _parse_known_args
    start_index = consume_optional(start_index)

  File "/usr/lib64/python3.15/argparse.py", line 2330, in consume_optional
    take_action(action, args, option_string)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/lib64/python3.15/argparse.py", line 2231, in take_action
    action(self, namespace, argument_values, option_string)
    ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/lib64/python3.15/argparse.py", line 1279, in __call__
    parser.print_help()
    ~~~~~~~~~~~~~~~~~^^

  File "/usr/lib64/python3.15/argparse.py", line 2882, in print_help
    help_text = self.format_help()

  File "/home/ben/src/forks/snakemake/.venv/lib64/python3.15/site-packages/configargparse.py", line 1696, in format_help
    return argparse.ArgumentParser.format_help(self) + (
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^

  File "/usr/lib64/python3.15/argparse.py", line 2839, in format_help
    return formatter.format_help()
           ~~~~~~~~~~~~~~~~~~~~~^^

  File "/usr/lib64/python3.15/argparse.py", line 338, in format_help
    help = self._root_section.format_help()

  File "/usr/lib64/python3.15/argparse.py", line 263, in format_help
    item_help = join([func(*args) for func, args in self.items])
                      ~~~~^^^^^^^

  File "/usr/lib64/python3.15/argparse.py", line 263, in format_help
    item_help = join([func(*args) for func, args in self.items])
                      ~~~~^^^^^^^

  File "/usr/lib64/python3.15/argparse.py", line 612, in _format_action
    help_text = self._expand_help(action)

  File "/usr/lib64/python3.15/argparse.py", line 712, in _expand_help
    help_string = self._get_help_string(action)

  File "/home/ben/src/forks/snakemake/src/snakemake/common/argparse.py", line 76, in _get_help_string
    and not isinstance(action.default, dataclasses._MISSING_TYPE)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^

AttributeError: module 'dataclasses' has no attribute '_MISSING_TYPE'

The TypeError is supposed to happen, but it is supposed to be caught: https://github.com/python/cpython/blob/f31a89bb901067dd105b00cfa90523cf7ffdbbdd/Lib/argparse.py#L2877-L2882; that’s all in the Python standard library. But then the AttributeError on _MISSING_TYPE happens, and that’s the bug/regression in Snakemake.

After this PR, snakemake --help works as expected. Trying again in a Python 3.14 virtualenv, I confirm that snakemake --help still works as expected.

There are probably 100 different ways that this PR could have been written differently, and half of them are actually reasonable. I’m happy to consider style and implementation changes if requested.

One thing I considered is that it might be just as good to always test against dataclasses.MISSING, which already existed with sentinel semantics even before the introduction of the PEP 661 sentinel type. I think that this probably works as expected, and it’s a lot simpler:

diff --git a/src/snakemake/common/argparse.py b/src/snakemake/common/argparse.py
index 1801419d..db8d8b07 100644
--- a/src/snakemake/common/argparse.py
+++ b/src/snakemake/common/argparse.py
@@ -73,7 +73,7 @@ class ArgumentDefaultsHelpFormatter(argparse.HelpFormatter):
                 or action.nargs in [argparse.OPTIONAL, argparse.ZERO_OR_MORE]
             )
             and action.default not in (None, "", set(), argparse.SUPPRESS)
-            and not isinstance(action.default, dataclasses._MISSING_TYPE)
+            and not action.default is dataclasses.MISSING
         ):
             if isinstance(action.default, collections.abc.Iterable) and not isinstance(
                 action.default, str

It’s worth considering that both of these approaches rely on implementation details of dataclasses.MISSING, and either of them may break at any point in the future. See the warning in https://docs.python.org/3.14/library/dataclasses.html#dataclasses.field that “No code should directly use the MISSING value.” Maybe taking a step back and reflecting on the bigger picture could reveal a better approach that sidesteps the issue completely. I’m not prepared to spend any more time on it right now, so please take this PR as a combined bug report and initial suggestion.

QC

  • The PR contains a test case for the changes or the changes are already covered by an existing test case. To be honest, I don’t think there is anything in the test suite that checks that snakemake --help doesn’t crash, but I’m not prepared to contribute a test case. I welcome the addition of a relevant test case to this PR, but my own contribution will be limited to fixing the regression and confirming the fix manually.
  • The documentation (docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). not necessary
  • I, as a human being, have checked each line of code in this pull request

AI-assistance disclosure

I used AI assistance for:

  • Code generation (e.g., when writing an implementation or fixing a bug)
  • Test/benchmark generation
  • Documentation (including examples)
  • Research and understanding

I did not use any AI assistance at any point in preparing this PR.

Summary by CodeRabbit

  • Bug Fixes
    • Ensure command-line help consistently recognizes and displays missing/default argument values across Python versions, preventing incorrect omission or duplication of default values in help text.

@coderabbitai

coderabbitai Bot commented Jun 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d4e205ce-f3ee-4221-aad9-45ca61fad774

📥 Commits

Reviewing files that changed from the base of the PR and between 33670f4 and 6d13f11.

📒 Files selected for processing (1)
  • src/snakemake/common/argparse.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/snakemake/common/argparse.py
📝 Walkthrough

Walkthrough

The PR updates ArgumentDefaultsHelpFormatter in the argparse module to consistently handle dataclass "missing" sentinels across Python versions. A new _is_missing helper method supports both dataclasses.MISSING (Python 3.15+) and dataclasses._MISSING_TYPE (older versions), and _get_help_string is refactored to use this helper.

Changes

Argument help formatting compatibility

Layer / File(s) Summary
Missing sentinel helper
src/snakemake/common/argparse.py		 _is_missing static method now branches on dataclasses module attributes to detect missing sentinels in both newer and older Python versions, replacing the direct _MISSING_TYPE instance check in _get_help_string.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: adapting code to handle dataclasses._MISSING_TYPE being replaced with a sentinel in Python 3.15.
Description check ✅ Passed The description provides comprehensive context including links to CPython changes, a detailed traceback showing the bug, discussion of alternatives, and addresses all QC checklist items with appropriate reasoning.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@musicinmybrain musicinmybrain changed the title Adapt for dataclasses._MISSING_TYPE replaced with sentinel in Python 3.15 fix: Adapt for dataclasses._MISSING_TYPE replaced with sentinel in Python 3.15 Jun 1, 2026
johanneskoester
johanneskoester approved these changes Jun 1, 2026
View reviewed changes

@johanneskoester johanneskoester left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

@johanneskoester johanneskoester merged commit b7fb8df into snakemake:main Jun 1, 2026
90 checks passed
@snakemake-bot snakemake-bot mentioned this pull request Jun 1, 2026
Merged
johanneskoester pushed a commit that referenced this pull request Jun 1, 2026
@snakemake-bot
chore(main): release 9.22.0 (#4210)
6998250 
🤖 I have created a release *beep* *boop*
---


##
[9.22.0](v9.21.1...v9.22.0)
(2026-06-01)


### Features

* add semantic helper functions choose_file/choose_folder/choose_tmp and
allow all semantic helper functions to be used when parsing resources
([#3820](#3820))
([d3c2386](d3c2386))
* add temp to default pathvars
([#4108](#4108))
([917fb11](917fb11))
* add workflow info to log
([#4079](#4079))
([e40e15b](e40e15b))
* support python 3.14
([#3739](#3739))
([7e3be0c](7e3be0c))


### Bug Fixes

* Adapt for dataclasses._MISSING_TYPE replaced with sentinel in Python
3.15 ([#4211](#4211))
([b7fb8df](b7fb8df))
* ensure that storage is not actually retrieved with --touch
([#4212](#4212))
([2726cae](2726cae))
* symlink to directory "has older modification time" than target
([#3782](#3782))
([#3784](#3784))
([41903d8](41903d8))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@johanneskoester johanneskoester johanneskoester approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@musicinmybrain @johanneskoester