Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

functools' singledispatch does not support GenericAlias #90190

Closed
Tracked in #89828
kumaraditya303 opened this issue Dec 10, 2021 · 9 comments
Closed
Tracked in #89828

functools' singledispatch does not support GenericAlias #90190

kumaraditya303 opened this issue Dec 10, 2021 · 9 comments
Assignees
Labels
3.9 3.10 3.11 docs stdlib type-bug

Comments

@kumaraditya303
Copy link
Contributor

@kumaraditya303 kumaraditya303 commented Dec 10, 2021

BPO 46032
Nosy @rhettinger, @ambv, @serhiy-storchaka, @miss-islington, @uriyyo, @kumaraditya303, @AlexWaygood
PRs
  • #30050
  • #30254
  • #30255
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = <Date 2021-12-26.12:23:57.239>
    created_at = <Date 2021-12-10.09:04:56.137>
    labels = ['type-bug', '3.9', '3.10', '3.11', 'library', 'docs']
    title = "functools' singledispatch does not support GenericAlias"
    updated_at = <Date 2021-12-26.12:23:57.239>
    user = 'https://github.com/kumaraditya303'

    bugs.python.org fields:

    activity = <Date 2021-12-26.12:23:57.239>
    actor = 'serhiy.storchaka'
    assignee = 'serhiy.storchaka'
    closed = True
    closed_date = <Date 2021-12-26.12:23:57.239>
    closer = 'serhiy.storchaka'
    components = ['Documentation', 'Library (Lib)']
    creation = <Date 2021-12-10.09:04:56.137>
    creator = 'kumaraditya'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 46032
    keywords = ['patch']
    message_count = 9.0
    messages = ['408179', '408181', '408204', '408205', '408306', '408348', '409169', '409170', '409197']
    nosy_count = 7.0
    nosy_names = ['rhettinger', 'lukasz.langa', 'serhiy.storchaka', 'miss-islington', 'uriyyo', 'kumaraditya', 'AlexWaygood']
    pr_nums = ['30050', '30254', '30255']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue46032'
    versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    @kumaraditya303
    Copy link
    Contributor Author

    @kumaraditya303 kumaraditya303 commented Dec 10, 2021

    functools' singledispatch does not support GenericAlias

    from functools import singledispatch
    
    @singledispatch
    def func(x):
        print("any")
    
    @func.register
    def _(x: list[str]):
        print("list[str]")
    
    
    func(["a", "b"])

    @kumaraditya303 kumaraditya303 added type-feature 3.11 stdlib labels Dec 10, 2021
    @AlexWaygood
    Copy link
    Member

    @AlexWaygood AlexWaygood commented Dec 10, 2021

    My opinion is that supporting GenericAlias here would be a bad idea. Associating an implementation of the function with the argument type list[str] is ambiguous. Would this implementation be called if any argument of type list was supplied, or would it only be called if all elements in the list were of type str?

    The first option would be efficient, simple, and similar to the way singledispatch treats most other argument-types. However, it would be unintuitive.

    The second option would be more intuitive, but could be extremely inefficient if a very long list was passed in. It would also make the code more complicated.

    @AlexWaygood
    Copy link
    Member

    @AlexWaygood AlexWaygood commented Dec 10, 2021

    It would be well worth it to improve the error message, however:

    >>> from functools import singledispatch
    >>> @singledispatch
    ... def func(arg):
    ...     raise NotImplementedError
    ... 
    >>> @func.register
    ... def _(arg: list[str]):
    ...     print('Got a list of strings')
    ... 
    >>> func(1)
    Traceback (most recent call last):
      File "/usr/local/lib/python3.9/functools.py", line 830, in dispatch
        impl = dispatch_cache[cls]
      File "/usr/local/lib/python3.9/weakref.py", line 405, in __getitem__
        return self.data[ref(key)]
    KeyError: <weakref at 0x7f2a0d9141d0; to 'type' at 0x7f2a0e08b200 (int)>
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/local/lib/python3.9/functools.py", line 833, in dispatch
        impl = registry[cls]
    KeyError: <class 'int'>
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/local/lib/python3.9/functools.py", line 877, in wrapper
        return dispatch(args[0].__class__)(*args, **kw)
        return dispatch(args[0].__class__)(*args, **kw)
      File "/usr/local/lib/python3.9/functools.py", line 835, in dispatch
        impl = _find_impl(cls, registry)
      File "/usr/local/lib/python3.9/functools.py", line 782, in _find_impl
        mro = _compose_mro(cls, registry.keys())
      File "/usr/local/lib/python3.9/functools.py", line 743, in _compose_mro
        types = [n for n in types if is_related(n)]
      File "/usr/local/lib/python3.9/functools.py", line 743, in <listcomp>
        types = [n for n in types if is_related(n)]
      File "/usr/local/lib/python3.9/functools.py", line 742, in is_related
        and issubclass(cls, typ))
    TypeError: issubclass() argument 2 cannot be a parameterized generic
    

    @AlexWaygood
    Copy link
    Member

    @AlexWaygood AlexWaygood commented Dec 10, 2021

    The above traceback is because the isinstance(list[str], type) check at Lib/functools.py:848 evaluates to True. Related: bpo-45665.

    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented Dec 11, 2021

    Yes, it is related to bpo-45665. It is a complicated case due to coincidence of several circumstances.

    1. isinstance(list[int], type) is True, while isinstance(typing.List[int], type) is False. list[int] is considered a type in this check.

    2. list[int].__mro__ == list.__mro__, while typing.List[int] does not have the __mro__ attribute. list[int] is considered a type in this check.

    3. issubclass(cls, list[int]) raises a TypeError (the same for typing.List[int]). list[int] cannot be used as a type here.

    4. 2-argument registry() does not check the type of its first argument. f.registry(42, ...) is silently passed.

    In 2-argument registry() typing.List[int] is passed due to (4) and ignored in dispatch() due to (2). list[int] is passed due to (4), but caused error due to (3).

    In other uses of registry() (1-argument decorator factory and decorator with annotations) typing.List[int] is not passed due to 1. list[int] is passed due to (1) and caused error due to (3).

    The proposed PR makes list[int] be treated the same way as typing.List[int]. It also makes 2-argument registry() rejecting invalid first argument, so all three forms of registry() accept and reject now the same types.

    @AlexWaygood
    Copy link
    Member

    @AlexWaygood AlexWaygood commented Dec 11, 2021

    The PR looks good to me. I think it's also important that we document that these types aren't supported, as it's not mentioned anywhere at the moment. Related: bpo-34498.

    @AlexWaygood AlexWaygood added docs type-bug and removed type-feature labels Dec 11, 2021
    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented Dec 25, 2021

    New changeset 078abb6 by Serhiy Storchaka in branch 'main':
    bpo-46032: Check types in singledispatch's register() at declaration time (GH-30050)
    078abb6

    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented Dec 25, 2021

    New changeset 03c7449 by Serhiy Storchaka in branch '3.10':
    [3.10] bpo-46032: Check types in singledispatch's register() at declaration time (GH-30050) (GH-30254)
    03c7449

    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented Dec 26, 2021

    New changeset 25a12aa by Miss Islington (bot) in branch '3.9':
    [3.9] bpo-46032: Check types in singledispatch's register() at declaration time (GH-30050) (GH-30254) (GH-30255)
    25a12aa

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 3.10 3.11 docs stdlib type-bug
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants