Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimise inspect.getattr_static #103193

Closed
AlexWaygood opened this issue Apr 2, 2023 · 3 comments
Closed

Optimise inspect.getattr_static #103193

AlexWaygood opened this issue Apr 2, 2023 · 3 comments
Labels
3.12 new features, bugs and security fixes performance Performance or resource usage stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@AlexWaygood
Copy link
Member

AlexWaygood commented Apr 2, 2023

Feature or enhancement

Following 6d59c9e, typing._ProtocolMeta.__instancecheck__ uses inspect.getattr_static in a tight loop, where it had previously used hasattr. This improves semantics in several highly desirable ways, but causes a considerable slowdown for _ProtocolMeta.__instancecheck__, as inspect.getattr_static is much slower than hasattr.

The performance hit to _ProtocolMeta.__instancecheck__ has already been mostly mitigated through several typing-specific optimisations that are tracked in this issue:

However, it would be good to also see if we can improve the performance of inspect.getattr_static. This will not only improve the performance of isinstance() checks against classes subclassing typing.Protocol. It will also improve the performance of all other tools that use inspect.getattr_static for introspection without side effects.

Linked PRs

@AlexWaygood AlexWaygood added type-feature A feature request or enhancement performance Performance or resource usage stdlib Python modules in the Lib dir 3.12 new features, bugs and security fixes labels Apr 2, 2023
AlexWaygood added a commit to AlexWaygood/cpython that referenced this issue Apr 2, 2023
@AlexWaygood
Copy link
Member Author

I can't see anything further that can be easily optimised here, but happy to reopen if somebody else can!

@sobolevn
Copy link
Member

sobolevn commented Apr 5, 2023

I've measured my ideas, but the results are close to noise.

Initial numbers:

type[Foo]                :  793 ±  1 ns 
Foo                      : 1852 ±  3 ns 
type[Bar]                :  791 ±  1 ns 
Bar                      : 1843 ±  2 ns 
WithParentClassX         : 2768 ±  2 ns 
Baz                      : 2712 ±  8 ns 
WithParentX              : 3611 ±  9 ns 
type[Missing]            : 3364 ± 13 ns 
Missing                  : 2749 ± 22 ns 
Slotted                  : 2363 ±  7 ns 
Method                   : 1863 ± 11 ns 
StMethod                 : 1863 ±  8 ns 
ClsMethod                : 1847 ±  4 ns 

After all changes:

type[Foo]                :  793 ±  2 ns 
Foo                      : 1833 ±  2 ns 
type[Bar]                :  793 ±  3 ns 
Bar                      : 1833 ±  5 ns 
WithParentClassX         : 2756 ±  4 ns 
Baz                      : 2701 ±  8 ns 
WithParentX              : 3696 ± 28 ns 
type[Missing]            : 3390 ± 41 ns 
Missing                  : 2703 ±  9 ns 
Slotted                  : 2312 ± 11 ns 
Method                   : 1860 ± 19 ns 
StMethod                 : 1828 ±  5 ns 
ClsMethod                : 1859 ± 23 ns

Code:

from timeit import timeit
from statistics import mean, stdev
from inspect import getattr_static

class Foo:
    @property
    def x(self) -> int:
        return 42

class Bar:
    x = 42

class WithParentClassX(Bar): ...

class Baz:
    def __init__(self):
        self.x = 42

class WithParentX(Baz): ...

class Missing: ...

class Slotted:
    __slots__ = ('x',)
    def __init__(self):
        self.x = 42

class Method:
    def x(self): ...

class ClsMethod:
    @classmethod
    def x(cls): ...

class StMethod:
    @staticmethod
    def x(): ...

import gc
gc.disable()

times = []
def stats():
    ts = [t * 1e8 for t in sorted(times)[:5]]
    return f'{round(mean(ts)):4} ± {round(stdev(ts)):2} ns '

def bench(obj):
    # Warmup:
    for _ in range(5):
        number = 100
        timeit(lambda: getattr_static(obj, 'x', None), number=number)

    # Actual bench:
    for _ in range(50):
        number = 1000
        t = timeit(lambda: getattr_static(obj, 'x', None), number=number) / number
        times.append(t)

    bench_name = (
        f'type[{obj.__name__}]'
        if isinstance(obj, type)
        else obj.__class__.__name__
    )
    print(f"{bench_name: <25}: {stats()}")
    times.clear()


bench(Foo)
bench(Foo())
bench(Bar)
bench(Bar())
bench(WithParentClassX())
bench(Baz())
bench(WithParentX())
bench(Missing)
bench(Missing())
bench(Slotted())
bench(Method())
bench(StMethod())
bench(ClsMethod())

@AlexWaygood
Copy link
Member Author

I think I'm once again out of ideas, at least for now.

AlexWaygood added a commit to AlexWaygood/cpython that referenced this issue Apr 7, 2023
AlexWaygood added a commit that referenced this issue Apr 7, 2023
gaogaotiantian pushed a commit to gaogaotiantian/cpython that referenced this issue Apr 8, 2023
warsaw pushed a commit to warsaw/cpython that referenced this issue Apr 11, 2023
warsaw pushed a commit to warsaw/cpython that referenced this issue Apr 11, 2023
warsaw pushed a commit to warsaw/cpython that referenced this issue Apr 11, 2023
Improve performance of `inspect.getattr_static`
warsaw pushed a commit to warsaw/cpython that referenced this issue Apr 11, 2023
AlexWaygood added a commit to AlexWaygood/cpython that referenced this issue May 7, 2023
AlexWaygood added a commit that referenced this issue May 7, 2023
…r_static` (#104267)

Co-authored-by: Carl Meyer <carl@oddbird.net>
AlexWaygood added a commit to AlexWaygood/cpython that referenced this issue May 8, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 8, 2023
)

(cherry picked from commit 921185ed050efbca2f0adeab79f676b7f8cc3660)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
AlexWaygood added a commit that referenced this issue May 8, 2023
…104290)

gh-103193: Improve `getattr_static` test coverage (GH-104286)
(cherry picked from commit 921185e)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.12 new features, bugs and security fixes performance Performance or resource usage stdlib Python modules in the Lib dir type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

2 participants