Skip to content

Moved the datastream stop check poll to a separate thread to avoid blocking the event queue#925

Merged
jzakaryan merged 9 commits intolinkedin:masterfrom
jzakaryan:bmmRestart5
Feb 16, 2023
Merged

Moved the datastream stop check poll to a separate thread to avoid blocking the event queue#925
jzakaryan merged 9 commits intolinkedin:masterfrom
jzakaryan:bmmRestart5

Conversation

@jzakaryan
Copy link
Copy Markdown
Collaborator

  • After issuing a stop stream assignment, the leader polls the zookeeper to confirm that tasks have stopped. Currently that poll happens in the main event thread.
  • There is also a requirement that the coordinator should process all of its events in less than 30 seconds. If this requirement is not met the coordinator won't be able to punch heartbeats.
  • If the coordinator is configured to wait for stop propagation for a prolonged time, it will block the event queue for more than 30 seconds.

This change fixes the problem by moving the task to wait for stop request to propagate to a separate thread.

@jzakaryan jzakaryan requested a review from vmaheshw February 10, 2023 01:04
@jzakaryan jzakaryan requested a review from atoomula February 16, 2023 17:47
@jzakaryan jzakaryan merged commit 84f0e95 into linkedin:master Feb 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants