[Python] [Typeguard] Part 4 - Add Typeguard to AIO stack in tests by asheshvidyut · Pull Request #40226 · grpc/grpc

asheshvidyut · 2025-07-16T14:06:40Z

Description

Last Part of Add Typeguard to AIO stack - runs only in tests, fixes type hints for AIO Stack
Change -

Type Annotation Update in Channel Constructor

File: _channel.py (line 330)
Before: credentials: Optional[grpc.ChannelCredentials],
After: credentials: Optional[cygrpc.ChannelCredentials],

Change:
Updated type hint from grpc.ChannelCredentials to cygrpc.ChannelCredentials

Metadata Class Method Enhancement

File: _metadata.py (around line 100-102)
Before: def from_tuple(cls, raw_metadata: tuple):
After: def from_tuple(cls, raw_metadata: Union[tuple, Self]):

`GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY`decode at Cython layer.

File: src/python/grpcio/grpc/_cython/_cygrpc/records.pyx.pxi
Before: GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY = ( _GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY)
After: GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY = ( _GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY).decode()

Testing

CI

Related Work

Part 1: [Python][Typeguard] Part 1: Add Typeguard to AIO stack in tests #40201
Part 2: [Python][Typeguard] Part 2 - Add Typeguard to AIO stack in tests #40215
Part 3: [Python][Typeguard] Part 3 - Add Typeguard to AIO stack in tests #40217
Part 5: [Python] [Typeguard] Part 5 - Add Typeguard SYNC Stack in tests #40278
b/423755915

src/python/grpcio_tests/tests_aio/unit/_metadata_test.py

src/python/grpcio_tests/tests_aio/unit/auth_context_test.py

src/python/grpcio_tests/tests_aio/unit/channel_argument_test.py

src/python/grpcio_tests/tests_aio/unit/compression_test.py

src/python/grpcio_tests/tests_aio/unit/metadata_test.py

src/python/grpcio_tests/tests_aio/unit/outside_init_test.py

src/python/grpcio_tests/tests_aio/unit/secure_call_test.py

src/python/grpcio/grpc/aio/_call.py

src/python/grpcio/grpc/aio/_channel.py

src/python/grpcio/grpc/aio/_metadata.py

asheshvidyut · 2025-07-26T11:14:22Z

@sergiitk as discussed - I am only supressing the test cases which have *invalid* data now.

asheshvidyut · 2025-07-26T12:44:20Z

src/python/grpcio/grpc/aio/_metadata.py


    def get(
-        self, key: MetadataKey, default: MetadataValue = None
+        self, key: MetadataKey, default: Optional[MetadataValue] = None


Had to change this because some tests were failing with following stack trace

====================================================================== ERROR: test_delete_values (src.python.grpcio_tests.tests_aio.unit._metadata_test.TestTypeMetadata) ---------------------------------------------------------------------- Traceback (most recent call last): File ".src/python/grpcio_tests/tests_aio/unit/_metadata_test.py", line 186, in test_delete_values self.assertEqual(metadata.get("key1"), "other value 1") File ".src/python/grpcio/grpc/aio/_metadata.py", line 104, in get def get( File ".site-packages/typeguard/_functions.py", line 136, in check_argument_types check_type_internal(value, annotation, memo) File ".site-packages/typeguard/_checkers.py", line 779, in check_type_internal checker(value, origin_type, args, memo) File ".site-packages/typeguard/_checkers.py", line 428, in check_union raise TypeCheckError(f"did not match any element in the union:\n{formatted_errors}") typeguard.TypeCheckError: argument "default" (None) did not match any element in the union: str: is not an instance of str bytes: is not bytes-like ====================================================================== ERROR: test_get_item (src.python.grpcio_tests.tests_aio.unit._metadata_test.TestTypeMetadata) ---------------------------------------------------------------------- Traceback (most recent call last): File ".src/python/grpcio_tests/tests_aio/unit/_metadata_test.py", line 118, in test_get_item self.assertEqual(metadata.get("key"), "value1") File ".src/python/grpcio/grpc/aio/_metadata.py", line 104, in get def get( File ".site-packages/typeguard/_functions.py", line 136, in check_argument_types check_type_internal(value, annotation, memo) File ".site-packages/typeguard/_checkers.py", line 779, in check_type_internal checker(value, origin_type, args, memo) File ".site-packages/typeguard/_checkers.py", line 428, in check_union raise TypeCheckError(f"did not match any element in the union:\n{formatted_errors}") typeguard.TypeCheckError: argument "default" (None) did not match any element in the union: str: is not an instance of str bytes: is not bytes-like

Yes, this makes sense.

src/python/grpcio/grpc/aio/_metadata.py

…part-4

sergiitk · 2026-02-13T00:00:44Z

/gemini review

gemini-code-assist

Code Review

This pull request enables Typeguard for more AIO modules and fixes several type hints. The changes look good, especially the use of @typeguard.suppress_type_checks in tests that intentionally use invalid types. I've found one area for improvement regarding a type hint in _metadata.py that could be made more accurate to reflect the implementation and the author's intent as described in the PR description. I've left a couple of comments with suggestions.

src/python/grpcio/grpc/aio/_metadata.py

gemini-code-assist · 2026-02-13T00:04:04Z

src/python/grpcio_tests/tests_aio/unit/_metadata_test.py


    def test_metadata_from_tuple(self):
        scenarios = (
-            (None, Metadata()),


This test case for None input should be kept. The from_tuple method can handle None, and its type hint should be updated to Optional[...] to reflect this, as suggested in my other comment. Restoring this test case will ensure None continues to be handled correctly.

sergiitk · 2025-12-09T16:30:44Z

src/python/grpcio_tests/tests_aio/unit/channel_argument_test.py

+    @typeguard.suppress_type_checks
    async def test_invalid_client_args(self):
+        # This test works on invalid client_args which is expected
+        # to make typeguard fail, hence the decorator
+        # @typeguard.supress_type_checks is used


here and elsewhere: the comment can be as simple as this, no need for long explanation in the test itself

Suggested change

@typeguard.suppress_type_checks

async def test_invalid_client_args(self):

# This test works on invalid client_args which is expected

# to make typeguard fail, hence the decorator

# @typeguard.supress_type_checks is used

@typeguard.suppress_type_checks # testing negative cases

src/python/grpcio/grpc/aio/_call.py

src/python/grpcio_tests/tests_aio/unit/compression_test.py

src/python/grpcio_tests/tests_aio/unit/outside_init_test.py

src/python/grpcio/grpc/aio/_metadata.py

src/python/grpcio_tests/tests_aio/unit/metadata_test.py

sergiitk · 2026-02-17T07:14:47Z

src/python/grpcio/grpc/aio/_metadata.py


    def get(
-        self, key: MetadataKey, default: MetadataValue = None
+        self, key: MetadataKey, default: Optional[MetadataValue] = None


Yes, this makes sense.

sergiitk · 2026-02-17T07:24:37Z

src/python/grpcio/grpc/_compression.py

 def compression_algorithm_to_metadata(compression: grpc.Compression):
    return (
-        cygrpc.GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY,
+        grpc._common.decode(cygrpc.GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY),


For posterity, this fixes

====================================================================== ERROR: test_client_call_level_compression_allowed_compression (src.python.grpcio_tests.tests_aio.unit.compression_test.TestCompression.test_client_call_level_compression_allowed_compression) ---------------------------------------------------------------------- Traceback (most recent call last): File "./src/python/grpcio_tests/tests_aio/unit/_test_base.py", line 31, in wrapper return loop.run_until_complete(f(*args, **kwargs)) ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 725, in run_until_complete return future.result() ~~~~~~~~~~~~~^^ File "./src/python/grpcio_tests/tests_aio/unit/compression_test.py", line 157, in test_client_call_level_compression_allowed_compression call = multicallable(_REQUEST, compression=grpc.Compression.Deflate) File "./src/python/grpcio/grpc/aio/_channel.py", line 150, in __call__ metadata = self._init_metadata(metadata, compression) File "./src/python/grpcio/grpc/aio/_channel.py", line 131, in _init_metadata metadata = Metadata( *_compression.augment_metadata(metadata, compression) ) File "./src/python/grpcio/grpc/aio/_metadata.py", line 38, in __init__ def __init__(self, *args: Tuple[MetadataKey, MetadataValue]) -> None: ...<2 lines>... self.add(md_key, md_value) File "./site-packages/typeguard/_functions.py", line 137, in check_argument_types check_type_internal(value, annotation, memo) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^ File "./site-packages/typeguard/_checkers.py", line 951, in check_type_internal checker(value, origin_type, args, memo) ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "./site-packages/typeguard/_checkers.py", line 384, in check_tuple check_type_internal(element, element_type, memo) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "./site-packages/typeguard/_checkers.py", line 951, in check_type_internal checker(value, origin_type, args, memo) ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "./site-packages/typeguard/_checkers.py", line 400, in check_tuple check_type_internal(element, element_type, memo) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "./site-packages/typeguard/_checkers.py", line 956, in check_type_internal raise TypeCheckError(f"is not an instance of {qualified_name(origin_type)}") typeguard.TypeCheckError: item 0 of item 0 of argument "args" (tuple) is not an instance of str ====================================================================== ERROR: test_client_call_level_compression_baned_compression (src.python.grpcio_tests.tests_aio.unit.compression_test.TestCompression.test_client_call_level_compression_baned_compression) ---------------------------------------------------------------------- Traceback (most recent call last): File "./src/python/grpcio_tests/tests_aio/unit/_test_base.py", line 31, in wrapper return loop.run_until_complete(f(*args, **kwargs)) ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 725, in run_until_complete return future.result() ~~~~~~~~~~~~~^^ File "./src/python/grpcio_tests/tests_aio/unit/compression_test.py", line 147, in test_client_call_level_compression_baned_compression call = multicallable(_REQUEST, compression=grpc.Compression.Gzip) File "./src/python/grpcio/grpc/aio/_channel.py", line 150, in __call__ metadata = self._init_metadata(metadata, compression) File "./src/python/grpcio/grpc/aio/_channel.py", line 131, in _init_metadata metadata = Metadata( *_compression.augment_metadata(metadata, compression) ) File "./src/python/grpcio/grpc/aio/_metadata.py", line 38, in __init__ def __init__(self, *args: Tuple[MetadataKey, MetadataValue]) -> None: ...<2 lines>... self.add(md_key, md_value) File "./site-packages/typeguard/_functions.py", line 137, in check_argument_types check_type_internal(value, annotation, memo) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^ File "./site-packages/typeguard/_checkers.py", line 951, in check_type_internal checker(value, origin_type, args, memo) ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "./site-packages/typeguard/_checkers.py", line 384, in check_tuple check_type_internal(element, element_type, memo) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "./site-packages/typeguard/_checkers.py", line 951, in check_type_internal checker(value, origin_type, args, memo) ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "./site-packages/typeguard/_checkers.py", line 400, in check_tuple check_type_internal(element, element_type, memo) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "./site-packages/typeguard/_checkers.py", line 956, in check_type_internal raise TypeCheckError(f"is not an instance of {qualified_name(origin_type)}") typeguard.TypeCheckError: item 0 of item 0 of argument "args" (tuple) is not an instance of str ----------------------------------------------------------------------

sergiitk · 2026-02-17T07:43:56Z

src/python/grpcio/grpc/_compression.py

 def compression_algorithm_to_metadata(compression: grpc.Compression):
    return (
-        cygrpc.GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY,
+        grpc._common.decode(cygrpc.GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY),


Looking further into this, I recommend defining GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY as a string in

grpc/src/python/grpcio/grpc/_cython/_cygrpc/records.pyx.pxi

Lines 114 to 115 in 456a47b

GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY = (

_GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY)

Since we know that a metadata key should never be bytes, we should just do

GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY = ( _GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY ).decode()

agree, and IIRC I had looked into this and had a similar comment when I first reviewed this PR too. Sadly, looks like the comment disappeared after the force push. Anyways, we also have a similar situation with cygrpc.ChannelArgKey.primary_user_agent_string as I mentioned here: https://github.com/grpc/grpc/pull/40278/changes#r2247823758

Values coming from Core tend to be char* (equivalent to bytes) which is the cause of this problem. We should convert them to string when we use them in Python

sreenithi · 2026-02-15T16:27:45Z

src/python/grpcio/grpc/aio/_metadata.py


    @classmethod
-    def from_tuple(cls, raw_metadata: tuple):
+    def from_tuple(cls, raw_metadata: Union[tuple, Self]):


If this function is also accepting a Metadata object, we should probably add a check for the type and return the object itself directly.
The current logic works, but it will be creating a new Metadata object from the given Metadata object, which seems redundant.

This is actually a very good point. Not sure how we missed this. The name of the method is clearly from_tuple. Is there a test how it'll even work with non-empty Metadata object?

Also, Self is misused here:

Special type to represent the current enclosed class.
— https://docs.python.org/3.14/library/typing.html#typing.Self

The main intent for this is to:

Annotate self argument as self: Self

Indicate the method returns self (f.e. in chaining), and require subclasses from returning superclasses.

Indicate abstract factory methods that return cls()

I don't think this was ever intended to be used in arguments.

@asheshvidyut - we should fix this ASAP.

My bad, looked at the spec, and it's OK to use it in params:

https://typing.python.org/en/latest/spec/generics.html#use-in-parameter-types

Looks like I missed the last check you added.

Discussed the plan to address this IRL.

We'll revert raw_metadata type to tuple

We'll going create an internal _create classmethod that will support instantiating Metadata from

None

Metadata (Self)

Iterable[tuple[MetadataKey, MetadataValue]]

Internally, we'll replace all places that violate the original signature def from_tuple(cls, raw_metadata: tuple) to the new _create method

We'll make the method public with the same gRFC where we introduces type hints to the sync stack.

sreenithi · 2026-02-17T08:38:00Z

src/python/grpcio/grpc/_compression.py

 def compression_algorithm_to_metadata(compression: grpc.Compression):
    return (
-        cygrpc.GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY,
+        grpc._common.decode(cygrpc.GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY),


agree, and IIRC I had looked into this and had a similar comment when I first reviewed this PR too. Sadly, looks like the comment disappeared after the force push. Anyways, we also have a similar situation with cygrpc.ChannelArgKey.primary_user_agent_string as I mentioned here: https://github.com/grpc/grpc/pull/40278/changes#r2247823758

Values coming from Core tend to be char* (equivalent to bytes) which is the cause of this problem. We should convert them to string when we use them in Python

github-actions bot added the lang/Python label Jul 16, 2025

asheshvidyut added the release notes: no Indicates if PR should not be in release notes label Jul 16, 2025

grpc-checks bot added bloat/none per-call-memory/neutral per-channel-memory/neutral labels Jul 16, 2025

sergiitk self-assigned this Jul 24, 2025

asheshvidyut force-pushed the feature/type-hints/typeguard-aio-stack-part-4 branch 2 times, most recently from 26572b0 to c584ba8 Compare July 25, 2025 08:17

asheshvidyut requested review from sergiitk and sreenithi July 25, 2025 09:21

asheshvidyut commented Jul 25, 2025

View reviewed changes

asheshvidyut requested review from eugeneo and removed request for eugeneo July 25, 2025 09:49

asheshvidyut marked this pull request as ready for review July 25, 2025 10:49

asheshvidyut requested a review from eugeneo as a code owner July 25, 2025 10:49

asheshvidyut commented Jul 26, 2025

View reviewed changes

src/python/grpcio/grpc/aio/_call.py Outdated Show resolved Hide resolved

src/python/grpcio/grpc/aio/_channel.py Show resolved Hide resolved

asheshvidyut commented Jul 26, 2025

View reviewed changes

src/python/grpcio/grpc/aio/_metadata.py Outdated Show resolved Hide resolved

grpc-checks bot added per-call-memory/increase per-call-memory/neutral and removed per-call-memory/neutral per-call-memory/increase labels Jul 26, 2025

asheshvidyut commented Jul 26, 2025

View reviewed changes

src/python/grpcio/grpc/aio/_metadata.py Outdated Show resolved Hide resolved

asheshvidyut force-pushed the feature/type-hints/typeguard-aio-stack-part-4 branch from 5df8618 to d7cf624 Compare July 28, 2025 04:13

This was referenced Jul 30, 2025

[Python] [Typeguard] Part 5 - Add Typeguard SYNC Stack in tests #40278

Open

[Python] [Typeguard] Part 6 - Add Typeguard SYNC Stack in tests #40353

Open

asheshvidyut force-pushed the feature/type-hints/typeguard-aio-stack-part-4 branch from d7cf624 to f67aaa0 Compare July 30, 2025 11:08

asheshvidyut added this to the Python Type Hints Support milestone Aug 7, 2025

asheshvidyut force-pushed the feature/type-hints/typeguard-aio-stack-part-4 branch from f67aaa0 to 7fa0f84 Compare September 16, 2025 16:24

asheshvidyut added 17 commits January 26, 2026 14:33

remvoed suppression

6c8ea80

fix and removed some suppressions

a827cf0

rename test

1582c65

removed some suppression

1d5deb8

removed supression

305233f

removed some more suppressions

88738a5

removed unwanted imports

c61a4cc

isort and black code

9cf61f6

removed unsed import

e9dbf72

revert public method name change

fb7bf0a

fixes comment in asheshvidyut:feature/type-hints/typeguard-aio-stack-…

4e9d310

…part-4

removed the print statement

0dbc386

decode

76587bd

resolve comments

82e035d

fix other comments

c4a0dab

black code

d8db97d

fix typo

bbe3bf9

asheshvidyut force-pushed the feature/type-hints/typeguard-aio-stack-part-4 branch from a01f89b to bbe3bf9 Compare January 26, 2026 14:33

asheshvidyut mentioned this pull request Feb 12, 2026

[Python] Typehints generated code #41542

Open

gemini-code-assist bot reviewed Feb 13, 2026

View reviewed changes

sergiitk approved these changes Feb 17, 2026

View reviewed changes

sreenithi reviewed Feb 17, 2026

View reviewed changes

asheshvidyut added 4 commits February 17, 2026 10:22

update comment

53d754e

add check if already metadata

db6694d

decode in cython layer

c56e184

black

70db2d6

copybara-service bot closed this in 2b63f74 Feb 18, 2026

asheshvidyut mentioned this pull request Feb 19, 2026

[Python] New _create method for aio.Metadata #41678

Open

	GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY = (
	_GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY)

Conversation

asheshvidyut commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type Annotation Update in Channel Constructor

Metadata Class Method Enhancement

GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEYdecode at Cython layer.

Testing

Related Work

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

asheshvidyut commented Jul 26, 2025

Uh oh!

asheshvidyut Jul 26, 2025 • edited by sergiitk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sergiitk commented Feb 13, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

gemini-code-assist bot Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sergiitk Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

asheshvidyut commented Jul 16, 2025 •

edited

Loading

`GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY`decode at Cython layer.

asheshvidyut Jul 26, 2025 •

edited by sergiitk

Loading

sergiitk Feb 17, 2026 •

edited

Loading