Skip to content

[Python] Drop 3.9#42145

Closed
asheshvidyut wants to merge 100 commits into
grpc:masterfrom
asheshvidyut:feature/python/drop-3.9
Closed

[Python] Drop 3.9#42145
asheshvidyut wants to merge 100 commits into
grpc:masterfrom
asheshvidyut:feature/python/drop-3.9

Conversation

@asheshvidyut

@asheshvidyut asheshvidyut commented Apr 16, 2026

Copy link
Copy Markdown
Member

Description

Drop support for Python 3.9. Refer this policy for selecting the image.

References

Main PRs -

#34450
#38747

Follow up PRs -

#40139

Testing

CI

pawbhard and others added 12 commits April 23, 2026 08:45
…ngine_test (grpc#42185)

…ngine_test

TestResolveRemote, TestResolveIPv4Remote, and TestResolveIPv6Remote depend on external DNS services (localtest.me, nip.io, sslip.io). On Mac CI pool machines these lookups occasionally fail with kNotFound when the upstream resolver cannot reach the authoritative DNS servers, causing flaky test failures.

Add LookupWithRetry helper that retries up to 3 times on kNotFound. If all attempts fail the test is skipped rather than failed, since the failure is infrastructure unavailability not a code regression. Retrying only on kNotFound is safe: that status code is only produced by DNSServiceResolverImpl when the DNS server responds with NXDOMAIN for both A and AAAA; bugs in the resolver itself map to kUnknown and will still surface as failures.

Closes grpc#42185

COPYBARA_INTEGRATE_REVIEW=grpc#42185 from pawbhard:retry_dns 36bfcfd
PiperOrigin-RevId: 902619277
This will give us signal about the impact of ccache on the builds in the distribution tests. The code is copied from grpc#42033.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes grpc#42077

COPYBARA_INTEGRATE_REVIEW=grpc#42077 from murgatroid99:distribtest_ccache_stats ba85091
PiperOrigin-RevId: 902675367
…try to ccache download (grpc#42180)

### Description

Fix flaky tests for the log, looks like a network issue.

```
#18 [15/18] RUN curl -sSL -o ccache.tar.gz https://github.com/ccache/ccache/releases/download/v4.11.3/ccache-4.11.3.tar.gz     && tar -zxf ccache.tar.gz     && cd ccache-4.11.3     && mkdir build && cd build     && cmake -DCMAKE_BUILD_TYPE=Release -DZSTD_FROM_INTERNET=ON -DHIREDIS_FROM_INTERNET=ON ..     && make -j4 && make install     && cd ../..     && rm -rf ccache-4.11.3 ccache.tar.gz
#18 11.37
#18 11.37 gzip: stdin: not in gzip format
#18 11.37 tar: Child returned status 1
#18 11.37 tar: Error is not recoverable: exiting now
#18 ERROR: process "/bin/sh -c curl -sSL -o ccache.tar.gz https://github.com/ccache/ccache/releases/download/v4.11.3/ccache-4.11.3.tar.gz     && tar -zxf ccache.tar.gz     && cd ccache-4.11.3     && mkdir build && cd build     && cmake -DCMAKE_BUILD_TYPE=Release -DZSTD_FROM_INTERNET=ON -DHIREDIS_FROM_INTERNET=ON ..     && make -j4 && make install     && cd ../..     && rm -rf ccache-4.11.3 ccache.tar.gz" did not complete successfully: exit code: 2
------
```

Added retry to ccache install.

Closes grpc#42180

COPYBARA_INTEGRATE_REVIEW=grpc#42180 from asheshvidyut:fixit/fix-flake 33a66ed
PiperOrigin-RevId: 902683742
…pc#42150)

We enabled EventEngine for Python in the 1.80 release, making that the first time that any of these experiments were fully rolled out in open source. We plan to monitor for user issues for two full releases and then start removing the experiments. The new end date is the approximate target date for when we will do that.

Closes grpc#42150

COPYBARA_INTEGRATE_REVIEW=grpc#42150 from murgatroid99:extend_event_engine_experiments 72d2c0b
PiperOrigin-RevId: 902687984
Previously ccache wasn't properly enabled for our python builds before because setuptools has special logic to find MSVC and disregard `CC/CXX` flags https://github.com/pypa/setuptools/blob/main/setuptools/msvc.py. There's no graceful solution other than monkeypatching (which we already do with `_spawn_patch.py`).

Certain MSVC flags are not properly handled by ccache 4.8 so we need to rewrite and exclude them from hash key generation.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes grpc#42171

COPYBARA_INTEGRATE_REVIEW=grpc#42171 from yuanweiz:fixit-windows-ccache 5ff00ca
PiperOrigin-RevId: 902859436
The only github.com links in `build_autogenerated.yaml` are the ones that correspond to packages that `CMakeLists.txt` downloads, so we mirror those links in GCS the same way we do bazel dependencies. The goal is to use this mirror as an alternative download target in CMake to minimize flakes from having our downloads throttled by GitHub.

I also switched from `gsutil` to `gcloud storage` because `gsutil` wasn't working for me and the command line output recommended switching.

Closes grpc#42172

COPYBARA_INTEGRATE_REVIEW=grpc#42172 from murgatroid99:cmake_deps_mirror 63468be
PiperOrigin-RevId: 903662332
…dows version bump) (grpc#42195)

In the Python Windows builds, ccache currently has a low cache hit rate and fails to populate the direct mode cache. This is an attempt to fix that.

Closes grpc#42195

COPYBARA_INTEGRATE_REVIEW=grpc#42195 from murgatroid99:python_windows_ccache_fix 2205915
PiperOrigin-RevId: 903665805
…7.x (grpc#41948)

### Description

Remove upper bound from `grpc-status` `setup.py` install requirement for
protobuf.

Fixes grpc#41933 

### Testing

`cd grpc && bash test.sh`

```bash
VENV_NAME=".venv_status_pb_test"
python3 -m venv "$VENV_NAME"
source "$VENV_NAME/bin/activate"
pip install -U "protobuf>=7.0.0"
pip install -U googleapis-common-protos coverage
pip install -e .
cd src/python/grpcio_status
pip install --no-deps -e .
cd ../../..
export PYTHONPATH=$(pwd)/src/python/grpcio_tests:$(pwd)/src/python/grpcio_status
python3 -m unittest tests.status._grpc_status_test
python3 -m unittest tests_aio.status.grpc_status_test
deactivate
```

---------

Co-authored-by: Sergii Tkachenko <sergiitk@google.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…2198)

cmake/build/round_robin_test --gtest_filter=RoundRobinTest.StartsConnectingFromRandomIndex

Root Cause: we have 3 index 0,1, 2 and we query to get starting index. We expect to get a random index, so test assert we get something other than 0. the probability is 1/3 we do this 10 times , so failure probability becomes (1/3)^10 approx 0.0017%  . That is the reason we have not seen many instance of this failure.

Increasing the loop count to 30. (This will not increase test time as we will return early as soon as we get index other than 0). This will have failure probability (1/3)^30 ~ 10^ -15 (almost impossible)

Closes grpc#42198

COPYBARA_INTEGRATE_REVIEW=grpc#42198 from pawbhard:rr_flake 9a71e0b
PiperOrigin-RevId: 903699099
Closes grpc#42201

COPYBARA_INTEGRATE_REVIEW=grpc#42201 from rishesh007:p_fix d410156
PiperOrigin-RevId: 903774639
… build directory consistent across runs) (grpc#42212)

We see ccache cache misses because each python build runs in a different temporary directory, and some header file names in the cache entry's manifest appear to use the full file path, so they fail to match. This change resolves that by making the build directory consistent for each build type.

ref:
- b/501458064
- Previous change: grpc#42195

Closes grpc#42212

COPYBARA_INTEGRATE_REVIEW=grpc#42212 from murgatroid99:python_windows_ccache_basedir a29d1bb
PiperOrigin-RevId: 904139620
@asheshvidyut asheshvidyut requested a review from sreenithi April 23, 2026 03:41

@sreenithi sreenithi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just one comment regarding scl, but it's fine to do it in a follow up PR.

# the build. This differs from the main branch/legacy setup where artifact
# builds are usually run inside specific Docker images (managed by run_tests.py)
# that already have the correct environment enabled.
scl enable devtoolset-10 -- tools/run_tests/artifacts/build_artifact_python.sh

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ya that's expected if you remove both lines. I meant to say, add export PATH="/opt/rh/devtoolset-10/root/usr/bin:$PATH" back without removing it. And then you can remove scl enable... and it should work.

# the build. This differs from the main branch/legacy setup where artifact
# builds are usually run inside specific Docker images (managed by run_tests.py)
# that already have the correct environment enabled.
scl enable devtoolset-10 -- tools/run_tests/artifacts/build_artifact_python.sh

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But maybe to not block this PR from submission, i'm okay to do this in the follow up PR along with example updates.

asheshvidyut added a commit to a-detiste/grpc that referenced this pull request Jun 10, 2026
# Description

Drop support for Python `3.9`

# References

Main PRs -

grpc#34450
grpc#38747

Follow up PRs -

grpc#40139

# Testing

CI

Closes grpc#42145

PiperOrigin-RevId: 904381292
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.