[EE] Reinstate the backup poller for Python#39131
Closed
drfloob wants to merge 3 commits intogrpc:masterfrom
Closed
[EE] Reinstate the backup poller for Python#39131drfloob wants to merge 3 commits intogrpc:masterfrom
drfloob wants to merge 3 commits intogrpc:masterfrom
Conversation
yashykt
reviewed
Apr 2, 2025
| void grpc_client_channel_global_init_backup_polling() { | ||
| // Disable backup polling if EventEngine is used everywhere. | ||
| #ifdef GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER | ||
| g_backup_polling_disabled = false; |
Member
There was a problem hiding this comment.
note that we also have the experiment event_engine_callback_cq which was added in 1.69. If that is enabled, grpc_iomgr_run_in_background will return true, and the backup poller will not be started. Why not add this logic in grpc_client_channel_start_backup_polling/grpc_client_channel_stop_backup_polling instead?
Member
Author
There was a problem hiding this comment.
Done. If GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER is defined, the all of the logic to disable the backup poller is removed at compile time.
|
IIUC this a workaround for #39113 . |
sreenithi
approved these changes
May 28, 2025
This was referenced May 30, 2025
copybara-service bot
pushed a commit
that referenced
this pull request
Jun 20, 2025
…en EE is disabled (#39894) Fix the issue with gRPC Python Client not recovering in certain situations: #38290, #39113, #39631. --- This PR: - Keeps iomgr backup poller enabled on platforms that do not support that have EventEngine / POSIX poller disabled. - Fixes and hardens Python's reconnect_test. This fixes issues with Python Client recovery in certain cases when it falls back to the backup poller. Note that currently Python disables POSIX poller manually via `GRPC_PLATFORM_SUPPORTS_POSIX_POLLING` due to currently unresolved fork issue. Unlike #39131, this change is meant to be tested and merged to master, and then backported to past branches with broken python client reconnect logic. However, once Event Engine fork support is fixed, Python will switch to it by removing `GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER` from all builds. After that, the backup poller should not be needed for Python. With the updated test, the case with python relying on the backup poller is covered, and we should be able to detect issues with Python having to rely on in after switching to EE. Closes #39894 COPYBARA_INTEGRATE_REVIEW=#39894 from sergiitk:python-fix-reconnect 239b330 PiperOrigin-RevId: 773549056
sreenithi
pushed a commit
to sreenithi/grpc
that referenced
this pull request
Jun 23, 2025
…en EE is disabled (grpc#39894) Fix the issue with gRPC Python Client not recovering in certain situations: grpc#38290, grpc#39113, grpc#39631. --- This PR: - Keeps iomgr backup poller enabled on platforms that do not support that have EventEngine / POSIX poller disabled. - Fixes and hardens Python's reconnect_test. This fixes issues with Python Client recovery in certain cases when it falls back to the backup poller. Note that currently Python disables POSIX poller manually via `GRPC_PLATFORM_SUPPORTS_POSIX_POLLING` due to currently unresolved fork issue. Unlike grpc#39131, this change is meant to be tested and merged to master, and then backported to past branches with broken python client reconnect logic. However, once Event Engine fork support is fixed, Python will switch to it by removing `GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER` from all builds. After that, the backup poller should not be needed for Python. With the updated test, the case with python relying on the backup poller is covered, and we should be able to detect issues with Python having to rely on in after switching to EE. Closes grpc#39894 COPYBARA_INTEGRATE_REVIEW=grpc#39894 from sergiitk:python-fix-reconnect 239b330 PiperOrigin-RevId: 773549056
sreenithi
pushed a commit
to sreenithi/grpc
that referenced
this pull request
Jun 23, 2025
…en EE is disabled (grpc#39894) Fix the issue with gRPC Python Client not recovering in certain situations: grpc#38290, grpc#39113, grpc#39631. --- This PR: - Keeps iomgr backup poller enabled on platforms that do not support that have EventEngine / POSIX poller disabled. - Fixes and hardens Python's reconnect_test. This fixes issues with Python Client recovery in certain cases when it falls back to the backup poller. Note that currently Python disables POSIX poller manually via `GRPC_PLATFORM_SUPPORTS_POSIX_POLLING` due to currently unresolved fork issue. Unlike grpc#39131, this change is meant to be tested and merged to master, and then backported to past branches with broken python client reconnect logic. However, once Event Engine fork support is fixed, Python will switch to it by removing `GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER` from all builds. After that, the backup poller should not be needed for Python. With the updated test, the case with python relying on the backup poller is covered, and we should be able to detect issues with Python having to rely on in after switching to EE. Closes grpc#39894 COPYBARA_INTEGRATE_REVIEW=grpc#39894 from sergiitk:python-fix-reconnect 239b330 PiperOrigin-RevId: 773549056
sreenithi
pushed a commit
to sreenithi/grpc
that referenced
this pull request
Jun 23, 2025
…en EE is disabled (grpc#39894) Fix the issue with gRPC Python Client not recovering in certain situations: grpc#38290, grpc#39113, grpc#39631. --- This PR: - Keeps iomgr backup poller enabled on platforms that do not support that have EventEngine / POSIX poller disabled. - Fixes and hardens Python's reconnect_test. This fixes issues with Python Client recovery in certain cases when it falls back to the backup poller. Note that currently Python disables POSIX poller manually via `GRPC_PLATFORM_SUPPORTS_POSIX_POLLING` due to currently unresolved fork issue. Unlike grpc#39131, this change is meant to be tested and merged to master, and then backported to past branches with broken python client reconnect logic. However, once Event Engine fork support is fixed, Python will switch to it by removing `GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER` from all builds. After that, the backup poller should not be needed for Python. With the updated test, the case with python relying on the backup poller is covered, and we should be able to detect issues with Python having to rely on in after switching to EE. Closes grpc#39894 COPYBARA_INTEGRATE_REVIEW=grpc#39894 from sergiitk:python-fix-reconnect 239b330 PiperOrigin-RevId: 773549056
sreenithi
pushed a commit
to sreenithi/grpc
that referenced
this pull request
Jun 23, 2025
…en EE is disabled (grpc#39894) Fix the issue with gRPC Python Client not recovering in certain situations: grpc#38290, grpc#39113, grpc#39631. --- This PR: - Keeps iomgr backup poller enabled on platforms that do not support that have EventEngine / POSIX poller disabled. - Fixes and hardens Python's reconnect_test. This fixes issues with Python Client recovery in certain cases when it falls back to the backup poller. Note that currently Python disables POSIX poller manually via `GRPC_PLATFORM_SUPPORTS_POSIX_POLLING` due to currently unresolved fork issue. Unlike grpc#39131, this change is meant to be tested and merged to master, and then backported to past branches with broken python client reconnect logic. However, once Event Engine fork support is fixed, Python will switch to it by removing `GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER` from all builds. After that, the backup poller should not be needed for Python. With the updated test, the case with python relying on the backup poller is covered, and we should be able to detect issues with Python having to rely on in after switching to EE. Closes grpc#39894 COPYBARA_INTEGRATE_REVIEW=grpc#39894 from sergiitk:python-fix-reconnect 239b330 PiperOrigin-RevId: 773549056
This was referenced Jun 23, 2025
yashykt
pushed a commit
that referenced
this pull request
Jun 23, 2025
…sues: use iomgr backup poller when EE is disabled (v1.73.x backport) (#39950) Backport of #39894 to v1.73.x. --- Fix the issue with gRPC Python Client not recovering in certain situations: #38290, #39113, #39631. --- This PR: - Keeps iomgr backup poller enabled on platforms that do not support that have EventEngine / POSIX poller disabled. - Fixes and hardens Python's reconnect_test. This fixes issues with Python Client recovery in certain cases when it falls back to the backup poller. Note that currently Python disables POSIX poller manually via `GRPC_PLATFORM_SUPPORTS_POSIX_POLLING` due to currently unresolved fork issue. Unlike #39131, this change is meant to be tested and merged to master, and then backported to past branches with broken python client reconnect logic. However, once Event Engine fork support is fixed, Python will switch to it by removing `GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER` from all builds. After that, the backup poller should not be needed for Python. With the updated test, the case with python relying on the backup poller is covered, and we should be able to detect issues with Python having to rely on in after switching to EE. Co-authored-by: Sergii Tkachenko <sergiitk@google.com>
anniefrchz
pushed a commit
to anniefrchz/grpc
that referenced
this pull request
Jun 25, 2025
…en EE is disabled (grpc#39894) Fix the issue with gRPC Python Client not recovering in certain situations: grpc#38290, grpc#39113, grpc#39631. --- This PR: - Keeps iomgr backup poller enabled on platforms that do not support that have EventEngine / POSIX poller disabled. - Fixes and hardens Python's reconnect_test. This fixes issues with Python Client recovery in certain cases when it falls back to the backup poller. Note that currently Python disables POSIX poller manually via `GRPC_PLATFORM_SUPPORTS_POSIX_POLLING` due to currently unresolved fork issue. Unlike grpc#39131, this change is meant to be tested and merged to master, and then backported to past branches with broken python client reconnect logic. However, once Event Engine fork support is fixed, Python will switch to it by removing `GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER` from all builds. After that, the backup poller should not be needed for Python. With the updated test, the case with python relying on the backup poller is covered, and we should be able to detect issues with Python having to rely on in after switching to EE. Closes grpc#39894 COPYBARA_INTEGRATE_REVIEW=grpc#39894 from sergiitk:python-fix-reconnect 239b330 PiperOrigin-RevId: 773549056
sergiitk
added a commit
that referenced
this pull request
Jun 26, 2025
…sues: use iomgr backup poller when EE is disabled (v1.72.x backport) (#39949) Backport of #39894 to v1.72.x. --- Fix the issue with gRPC Python Client not recovering in certain situations: #38290, #39113, #39631. --- This PR: - Keeps iomgr backup poller enabled on platforms that do not support that have EventEngine / POSIX poller disabled. - Fixes and hardens Python's reconnect_test. This fixes issues with Python Client recovery in certain cases when it falls back to the backup poller. Note that currently Python disables POSIX poller manually via `GRPC_PLATFORM_SUPPORTS_POSIX_POLLING` due to currently unresolved fork issue. Unlike #39131, this change is meant to be tested and merged to master, and then backported to past branches with broken python client reconnect logic. However, once Event Engine fork support is fixed, Python will switch to it by removing `GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER` from all builds. After that, the backup poller should not be needed for Python. With the updated test, the case with python relying on the backup poller is covered, and we should be able to detect issues with Python having to rely on in after switching to EE. Co-authored-by: Sergii Tkachenko <sergiitk@google.com>
sergiitk
added a commit
that referenced
this pull request
Jun 26, 2025
…sues: use iomgr backup poller when EE is disabled (#39948) Backport of #39894 to v1.71.x. --- Fix the issue with gRPC Python Client not recovering in certain situations: #38290, #39113, #39631. --- This PR: - Keeps iomgr backup poller enabled on platforms that do not support that have EventEngine / POSIX poller disabled. - Fixes and hardens Python's reconnect_test. This fixes issues with Python Client recovery in certain cases when it falls back to the backup poller. Note that currently Python disables POSIX poller manually via `GRPC_PLATFORM_SUPPORTS_POSIX_POLLING` due to currently unresolved fork issue. Unlike #39131, this change is meant to be tested and merged to master, and then backported to past branches with broken python client reconnect logic. However, once Event Engine fork support is fixed, Python will switch to it by removing `GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER` from all builds. After that, the backup poller should not be needed for Python. With the updated test, the case with python relying on the backup poller is covered, and we should be able to detect issues with Python having to rely on in after switching to EE. Co-authored-by: Sergii Tkachenko <sergiitk@google.com>
sergiitk
added a commit
to sergiitk/grpc
that referenced
this pull request
Jun 26, 2025
…en EE is disabled (grpc#39894) Fix the issue with gRPC Python Client not recovering in certain situations: grpc#38290, grpc#39113, grpc#39631. --- This PR: - Keeps iomgr backup poller enabled on platforms that do not support that have EventEngine / POSIX poller disabled. - Fixes and hardens Python's reconnect_test. This fixes issues with Python Client recovery in certain cases when it falls back to the backup poller. Note that currently Python disables POSIX poller manually via `GRPC_PLATFORM_SUPPORTS_POSIX_POLLING` due to currently unresolved fork issue. Unlike grpc#39131, this change is meant to be tested and merged to master, and then backported to past branches with broken python client reconnect logic. However, once Event Engine fork support is fixed, Python will switch to it by removing `GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER` from all builds. After that, the backup poller should not be needed for Python. With the updated test, the case with python relying on the backup poller is covered, and we should be able to detect issues with Python having to rely on in after switching to EE. Closes grpc#39894 COPYBARA_INTEGRATE_REVIEW=grpc#39894 from sergiitk:python-fix-reconnect 239b330 PiperOrigin-RevId: 773549056
paulosjca
pushed a commit
to paulosjca/grpc
that referenced
this pull request
Aug 23, 2025
…en EE is disabled (grpc#39894) Fix the issue with gRPC Python Client not recovering in certain situations: grpc#38290, grpc#39113, grpc#39631. --- This PR: - Keeps iomgr backup poller enabled on platforms that do not support that have EventEngine / POSIX poller disabled. - Fixes and hardens Python's reconnect_test. This fixes issues with Python Client recovery in certain cases when it falls back to the backup poller. Note that currently Python disables POSIX poller manually via `GRPC_PLATFORM_SUPPORTS_POSIX_POLLING` due to currently unresolved fork issue. Unlike grpc#39131, this change is meant to be tested and merged to master, and then backported to past branches with broken python client reconnect logic. However, once Event Engine fork support is fixed, Python will switch to it by removing `GRPC_DO_NOT_INSTANTIATE_POSIX_POLLER` from all builds. After that, the backup poller should not be needed for Python. With the updated test, the case with python relying on the backup poller is covered, and we should be able to detect issues with Python having to rely on in after switching to EE. Closes grpc#39894 COPYBARA_INTEGRATE_REVIEW=grpc#39894 from sergiitk:python-fix-reconnect 239b330 PiperOrigin-RevId: 773549056
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.