Skip to content

[Draft] C++ repro example for Python's ping timeout issue#39354

Draft
sreenithi wants to merge 1 commit intogrpc:masterfrom
sreenithi:ping_timeout_cpp_repro
Draft

[Draft] C++ repro example for Python's ping timeout issue#39354
sreenithi wants to merge 1 commit intogrpc:masterfrom
sreenithi:ping_timeout_cpp_repro

Conversation

@sreenithi
Copy link
Copy Markdown
Contributor

@sreenithi sreenithi commented Apr 23, 2025

The root cause of the ping timeout issue in Python that was recently reported by Dataflow as well as in issue #38282 was because Python has explicitly disabled the use of EventEngine polling using the GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER build flag (Reference: #33279).
Removing the flag solves this issue, however, considering that it originally was added because of lack of fork support when using EventEngine, the current work in progress to add fork support using EventEngine must be completed before removing this flag.

In addition, it was also found that the issue could not be reproduced when running via Bazel. This was because the GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER flag was not getting propagated upstream to the Core layer, unless specifically defined as a compiler option in tools/bazel.rc or passed in via command line

This PR hence includes the C++ server and client code to reproduce the ping timeout issue.

Steps to reproduce the ping timeout via Bazel in CPP:

  • export GRPC_TRACE=tcp,http,http2_keepalive,http2_ping,polling,polling_api
  • export GRPC_VERBOSITY=debug
  • Run bazel run //examples/cpp/ping_timeout_repro:greeter_server --copt=-DGRPC_DO_NOT_INSTANTIATE_POSIX_POLLER to start the CPP server
  • Run bazel run //examples/cpp/ping_timeout_repro:repro_greeter_client --copt=-DGRPC_DO_NOT_INSTANTIATE_POSIX_POLLER to start the CPP client

NOTE: The issue can only be reproduced when including the --copt flag via command line. Running the example as described above with the specific debug trace logs should show log lines such as:

I0000 00:00:1745420380.475681 3473830 chttp2_transport.cc:1963] ipv6:%5B::1%5D:50051: Ping timeout. Closing transport.
...
I0000 00:00:1745420380.476151 3473830 chttp2_transport.cc:1844] 0x7ffb90001600 CANCEL PINGS: UNKNOWN:ping timeout {grpc_status:14}
...
I0000 00:00:1745420380.477078 3473830 chttp2_transport.cc:1844] 0x7ffb90001600 CANCEL PINGS: UNKNOWN:goaway sent
...
I0000 00:00:1745420380.477298 3473830 ev_posix.cc:186] (polling-api) fd_shutdown(5)
...
I0000 00:00:1745420380.477517 3473830 tcp_posix.cc:1105] TCP:0x55a33ded8300 got_read: UNKNOWN:FD Shutdown {children:[UNAVAILABLE:endpoint shutdown]}

@sreenithi sreenithi added the release notes: no Indicates if PR should not be in release notes label Apr 23, 2025
@sreenithi sreenithi changed the title added CPP repro example for ping timeout issue [Draft] C++ repro example for Python's ping timeout issue Apr 23, 2025
@eugeneo eugeneo assigned drfloob and unassigned eugeneo Apr 23, 2025
@drfloob
Copy link
Copy Markdown
Member

drfloob commented Apr 23, 2025

Eugene, as you're driving the Fork support effort, you control the timeline for this. Please manage it.

@eugeneo
Copy link
Copy Markdown
Contributor

eugeneo commented Apr 24, 2025

I will keep it here till I have some time between fork support reviews, will test it on top of my patch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lang/c++ release notes: no Indicates if PR should not be in release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants