Skip to content

heads up: string enums in dicts/lists are serialized differently in Python 3.15 #744

@befeleme

Description

@befeleme

While building cattrs with Python 3.15.0a7 in Fedora Linux (in our yearly integration effort), test_stdlib_json and test_stdlib_json_converter fail.
Upon closer look, there's a difference in how Python 3.14 and Pytthon 3.15 handle string enum values stored in other types. See:

Python 3.15.0a7

>>> import json
>>> from enum import Enum
>>> class AStringEnum(str, Enum):
...     A = "a"
...     
>>> val = AStringEnum.A
>>> json.dumps(val)
'"a"'
>>> in_dict = json.dumps({"key": val})
>>> in_dict
'{"key": "AStringEnum.A"}'
>>> parsed = json.loads(in_dict)
>>> type(parsed['key'])
<class 'str'>
>>> repr(parsed['key'])
"'AStringEnum.A'"
>>> AStringEnum(parsed['key'])
Traceback (most recent call last):
  File "<python-input-15>", line 1, in <module>
    AStringEnum(parsed['key'])
    ~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.15/enum.py", line 732, in __call__
    return cls.__new__(cls, value)
           ~~~~~~~~~~~^^^^^^^^^^^^
  File "/usr/lib64/python3.15/enum.py", line 1221, in __new__
    raise ve_exc
ValueError: 'AStringEnum.A' is not a valid AStringEnum

Python 3.14.3

>>> import json
>>> from enum import Enum
>>> class AStringEnum(str, Enum):
...     A = "a"
...  
>>> val = AStringEnum.A
>>> json.dumps(val)
'"a"'
>>> json.dumps({"key": val})
'{"key": "a"}'
>>> in_dict = json.dumps({"key": val})
>>> parsed = json.loads(in_dict)
>>> type(parsed['key'])
<class 'str'>
>>> repr(parsed['key'])
"'a'"
>>> AStringEnum(parsed['key'])
<AStringEnum.A: 'a'>
Very long actual traceback from the test

__________________________ test_stdlib_json_converter __________________________
[gw0] linux -- Python 3.15.0 /usr/bin/python3

  • Exception Group Traceback (most recent call last):
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/cattrs-26.1.0/tests/test_preconf.py", line 299, in test_stdlib_json_converter
    | def test_stdlib_json_converter(everything: Everything):
    | ^^^
    | File "/usr/lib/python3.15/site-packages/hypothesis/core.py", line 1787, in wrapped_test
    | raise the_error_hypothesis_found
    | ExceptionGroup: Hypothesis found 4 distinct failures. (4 sub-exceptions)
    +-+---------------- 1 ----------------
    | Exception Group Traceback (most recent call last):
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/cattrs-26.1.0/tests/test_preconf.py", line 301, in test_stdlib_json_converter
    | assert converter.loads(converter.dumps(everything), Everything) == everything
    | ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/BUILDROOT/usr/lib/python3.15/site-packages/cattrs/preconf/json.py", line 26, in loads
    | return self.structure(loads(data, **kwargs), cl)
    | ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/BUILDROOT/usr/lib/python3.15/site-packages/cattrs/converters.py", line 591, in structure
    | return self._structure_func.dispatch(cl)(obj, cl)
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
    | File "", line 154, in structure_Everything
    | if errors: raise __c_cve('While structuring ' + 'Everything', errors, __cl)
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | cattrs.errors.ClassValidationError: While structuring Everything (3 sub-exceptions)
    | Falsifying example: test_stdlib_json_converter(
    | everything=Everything(string='',
    | bytes=b'',
    | an_int=0,
    | a_float=0.0,
    | a_dict={},
    | a_bare_dict={},
    | a_list=[],
    | a_homogenous_tuple=(),
    | a_hetero_tuple=('', 0, 0.0),
    | a_counter=Counter(),
    | a_mapping={},
    | a_mutable_mapping={},
    | a_sequence=(),
    | a_mutable_sequence=[],
    | a_set=set(),
    | a_mutable_set=set(),
    | a_frozenset=frozenset(),
    | an_int_enum=<AnIntEnum.A: 1>,
    | a_str_enum=<AStringEnum.A: 'a'>,
    | a_bare_enum=Everything.ABareEnum.B,
    | a_datetime=datetime.datetime(2000, 1, 1, 0, 0, tzinfo=datetime.timezone.utc),
    | a_date=datetime.date(2000, 1, 1),
    | a_string_enum_dict={<AStringEnum.A: 'a'>: 0},
    | a_bytes_dict={},
    | native_union=0,
    | native_union_with_spillover='',
    | native_union_with_union_spillover='',
    | a_namedtuple=C(c=0.0),
    | a_literal=<AStringEnum.A: 'a'>,
    | a_literal_with_bare=1),
    | )
    +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    | File "", line 95, in structure_Everything
    | res['a_str_enum'] = __c_structure_a_str_enum(o['a_str_enum'])
    | ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
    | File "/usr/lib64/python3.15/enum.py", line 732, in call
    | return cls.new(cls, value)
    | ~~~~~~~~~~~^^^^^^^^^^^^
    | File "/usr/lib64/python3.15/enum.py", line 1221, in new
    | raise ve_exc
    | ValueError: 'AStringEnum.A' is not a valid Everything.AStringEnum
    | Structuring class Everything @ attribute a_str_enum
    +---------------- 2 ----------------
    | Exception Group Traceback (most recent call last):
    | File "", line 115, in structure_Everything
    | res['a_string_enum_dict'] = __c_structure_a_string_enum_dict(o['a_string_enum_dict'], __c_type_a_string_enum_dict)
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "", line 17, in structure_mapping
    | cattrs.errors.IterableValidationError: While structuring typing.Dict[tests.test_preconf.Everything.AStringEnum, int] (1 sub-exception)
    | Structuring class Everything @ attribute a_string_enum_dict
    +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    | File "", line 11, in structure_mapping
    | File "/usr/lib64/python3.15/enum.py", line 732, in call
    | return cls.new(cls, value)
    | ~~~~~~~~~~~^^^^^^^^^^^^
    | File "/usr/lib64/python3.15/enum.py", line 1221, in new
    | raise ve_exc
    | ValueError: 'AStringEnum.A' is not a valid Everything.AStringEnum
    | Structuring mapping key @ key 'AStringEnum.A'
    +------------------------------------
    +---------------- 3 ----------------
    | Traceback (most recent call last):
    | File "", line 145, in structure_Everything
    | res['a_literal'] = __c_structure_a_literal(o['a_literal'], __c_type_a_literal)
    | ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/BUILDROOT/usr/lib/python3.15/site-packages/cattrs/converters.py", line 720, in _structure_enum_literal
    | raise Exception(f"{val} not in literal {type}") from None
    | Exception: AStringEnum.A not in literal typing.Literal[1, <AStringEnum.A: 'a'>]
    | Structuring class Everything @ attribute a_literal
    +------------------------------------
    +---------------- 2 ----------------
    | Exception Group Traceback (most recent call last):
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/cattrs-26.1.0/tests/test_preconf.py", line 301, in test_stdlib_json_converter
    | assert converter.loads(converter.dumps(everything), Everything) == everything
    | ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/BUILDROOT/usr/lib/python3.15/site-packages/cattrs/preconf/json.py", line 26, in loads
    | return self.structure(loads(data, **kwargs), cl)
    | ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/BUILDROOT/usr/lib/python3.15/site-packages/cattrs/converters.py", line 591, in structure
    | return self._structure_func.dispatch(cl)(obj, cl)
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
    | File "", line 154, in structure_Everything
    | if errors: raise __c_cve('While structuring ' + 'Everything', errors, __cl)
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | cattrs.errors.ClassValidationError: While structuring Everything (2 sub-exceptions)
    | Falsifying example: test_stdlib_json_converter(
    | everything=Everything(string='',
    | bytes=b'',
    | an_int=0,
    | a_float=0.0,
    | a_dict={},
    | a_bare_dict={},
    | a_list=[],
    | a_homogenous_tuple=(),
    | a_hetero_tuple=('', 0, 0.0),
    | a_counter=Counter(),
    | a_mapping={},
    | a_mutable_mapping={},
    | a_sequence=(),
    | a_mutable_sequence=[],
    | a_set=set(),
    | a_mutable_set=set(),
    | a_frozenset=frozenset(),
    | an_int_enum=<AnIntEnum.A: 1>,
    | a_str_enum=<AStringEnum.A: 'a'>,
    | a_bare_enum=Everything.ABareEnum.B,
    | a_datetime=datetime.datetime(2000, 1, 1, 0, 0, tzinfo=datetime.timezone.utc),
    | a_date=datetime.date(2000, 1, 1),
    | a_string_enum_dict={<AStringEnum.A: 'a'>: 0},
    | a_bytes_dict={},
    | native_union=0,
    | native_union_with_spillover='',
    | native_union_with_union_spillover='',
    | a_namedtuple=C(c=0.0),
    | a_literal=1,
    | a_literal_with_bare=1), # or any other generated value
    | )
    +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    | File "", line 95, in structure_Everything
    | res['a_str_enum'] = __c_structure_a_str_enum(o['a_str_enum'])
    | ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
    | File "/usr/lib64/python3.15/enum.py", line 732, in call
    | return cls.new(cls, value)
    | ~~~~~~~~~~~^^^^^^^^^^^^
    | File "/usr/lib64/python3.15/enum.py", line 1221, in new
    | raise ve_exc
    | ValueError: 'AStringEnum.A' is not a valid Everything.AStringEnum
    | Structuring class Everything @ attribute a_str_enum
    +---------------- 2 ----------------
    | Exception Group Traceback (most recent call last):
    | File "", line 115, in structure_Everything
    | res['a_string_enum_dict'] = __c_structure_a_string_enum_dict(o['a_string_enum_dict'], __c_type_a_string_enum_dict)
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "", line 17, in structure_mapping
    | cattrs.errors.IterableValidationError: While structuring typing.Dict[tests.test_preconf.Everything.AStringEnum, int] (1 sub-exception)
    | Structuring class Everything @ attribute a_string_enum_dict
    +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    | File "", line 11, in structure_mapping
    | File "/usr/lib64/python3.15/enum.py", line 732, in call
    | return cls.new(cls, value)
    | ~~~~~~~~~~~^^^^^^^^^^^^
    | File "/usr/lib64/python3.15/enum.py", line 1221, in new
    | raise ve_exc
    | ValueError: 'AStringEnum.A' is not a valid Everything.AStringEnum
    | Structuring mapping key @ key 'AStringEnum.A'
    +------------------------------------
    +---------------- 3 ----------------
    | Exception Group Traceback (most recent call last):
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/cattrs-26.1.0/tests/test_preconf.py", line 301, in test_stdlib_json_converter
    | assert converter.loads(converter.dumps(everything), Everything) == everything
    | ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/BUILDROOT/usr/lib/python3.15/site-packages/cattrs/preconf/json.py", line 26, in loads
    | return self.structure(loads(data, **kwargs), cl)
    | ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/BUILDROOT/usr/lib/python3.15/site-packages/cattrs/converters.py", line 591, in structure
    | return self._structure_func.dispatch(cl)(obj, cl)
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
    | File "", line 154, in structure_Everything
    | if errors: raise __c_cve('While structuring ' + 'Everything', errors, __cl)
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | cattrs.errors.ClassValidationError: While structuring Everything (2 sub-exceptions)
    | Falsifying example: test_stdlib_json_converter(
    | everything=Everything(string='',
    | bytes=b'',
    | an_int=0,
    | a_float=0.0,
    | a_dict={},
    | a_bare_dict={},
    | a_list=[],
    | a_homogenous_tuple=(),
    | a_hetero_tuple=('', 0, 0.0),
    | a_counter=Counter(),
    | a_mapping={},
    | a_mutable_mapping={},
    | a_sequence=(),
    | a_mutable_sequence=[],
    | a_set=set(),
    | a_mutable_set=set(),
    | a_frozenset=frozenset(),
    | an_int_enum=<AnIntEnum.A: 1>,
    | a_str_enum=<AStringEnum.A: 'a'>,
    | a_bare_enum=Everything.ABareEnum.B,
    | a_datetime=datetime.datetime(2000, 1, 1, 0, 0, tzinfo=datetime.timezone.utc),
    | a_date=datetime.date(2000, 1, 1),
    | a_string_enum_dict={},
    | a_bytes_dict={},
    | native_union=0,
    | native_union_with_spillover='',
    | native_union_with_union_spillover='',
    | a_namedtuple=C(c=0.0),
    | a_literal=<AStringEnum.A: 'a'>,
    | a_literal_with_bare=1), # or any other generated value
    | )
    +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    | File "", line 95, in structure_Everything
    | res['a_str_enum'] = __c_structure_a_str_enum(o['a_str_enum'])
    | ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
    | File "/usr/lib64/python3.15/enum.py", line 732, in call
    | return cls.new(cls, value)
    | ~~~~~~~~~~~^^^^^^^^^^^^
    | File "/usr/lib64/python3.15/enum.py", line 1221, in new
    | raise ve_exc
    | ValueError: 'AStringEnum.A' is not a valid Everything.AStringEnum
    | Structuring class Everything @ attribute a_str_enum
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    | File "", line 145, in structure_Everything
    | res['a_literal'] = __c_structure_a_literal(o['a_literal'], __c_type_a_literal)
    | ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/BUILDROOT/usr/lib/python3.15/site-packages/cattrs/converters.py", line 720, in _structure_enum_literal
    | raise Exception(f"{val} not in literal {type}") from None
    | Exception: AStringEnum.A not in literal typing.Literal[1, <AStringEnum.A: 'a'>]
    | Structuring class Everything @ attribute a_literal
    +------------------------------------
    +---------------- 4 ----------------
    | Exception Group Traceback (most recent call last):
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/cattrs-26.1.0/tests/test_preconf.py", line 301, in test_stdlib_json_converter
    | assert converter.loads(converter.dumps(everything), Everything) == everything
    | ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/BUILDROOT/usr/lib/python3.15/site-packages/cattrs/preconf/json.py", line 26, in loads
    | return self.structure(loads(data, **kwargs), cl)
    | ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "/builddir/build/BUILD/python-cattrs-26.1.0-build/BUILDROOT/usr/lib/python3.15/site-packages/cattrs/converters.py", line 591, in structure
    | return self._structure_func.dispatch(cl)(obj, cl)
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
    | File "", line 154, in structure_Everything
    | if errors: raise __c_cve('While structuring ' + 'Everything', errors, __cl)
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | cattrs.errors.ClassValidationError: While structuring Everything (1 sub-exception)
    | Falsifying example: test_stdlib_json_converter(
    | everything=Everything(string='',
    | bytes=b'',
    | an_int=0,
    | a_float=0.0,
    | a_dict={},
    | a_bare_dict={},
    | a_list=[],
    | a_homogenous_tuple=(),
    | a_hetero_tuple=('', 0, 0.0),
    | a_counter=Counter(),
    | a_mapping={},
    | a_mutable_mapping={},
    | a_sequence=(),
    | a_mutable_sequence=[],
    | a_set=set(),
    | a_mutable_set=set(),
    | a_frozenset=frozenset(),
    | an_int_enum=<AnIntEnum.A: 1>,
    | a_str_enum=<AStringEnum.A: 'a'>,
    | a_bare_enum=Everything.ABareEnum.B,
    | a_datetime=datetime.datetime(2000, 1, 1, 0, 0, tzinfo=datetime.timezone.utc),
    | a_date=datetime.date(2000, 1, 1),
    | a_string_enum_dict={},
    | a_bytes_dict={},
    | native_union=0,
    | native_union_with_spillover='',
    | native_union_with_union_spillover='',
    | a_namedtuple=C(c=0.0),
    | a_literal=1,
    | a_literal_with_bare=1),
    | )
    +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    | File "", line 95, in structure_Everything
    | res['a_str_enum'] = __c_structure_a_str_enum(o['a_str_enum'])
    | ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
    | File "/usr/lib64/python3.15/enum.py", line 732, in call
    | return cls.new(cls, value)
    | ~~~~~~~~~~~^^^^^^^^^^^^
    | File "/usr/lib64/python3.15/enum.py", line 1221, in new
    | raise ve_exc
    | ValueError: 'AStringEnum.A' is not a valid Everything.AStringEnum
    | Structuring class Everything @ attribute a_str_enum
    +------------------------------------

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions