Skip to content

fixes: support closable health server to avoid being stuck#734

Merged
mattisonchao merged 9 commits intomainfrom
fix.server.stuck
Jul 4, 2025
Merged

fixes: support closable health server to avoid being stuck#734
mattisonchao merged 9 commits intomainfrom
fix.server.stuck

Conversation

@mattisonchao
Copy link
Copy Markdown
Member

Motivation

Supports closable health servers to avoid the GRPC server graceful shutdown getting stuck due to the client watch stream not being closed.

Modification

  • Introduce a component named ClosableHeathServer
  • Close the ClosableHeathServer when the server closes.
  • Add a test for that.

@mattisonchao mattisonchao self-assigned this Jul 3, 2025
@mattisonchao mattisonchao requested a review from Copilot July 3, 2025 16:04

This comment was marked as outdated.

mattisonchao and others added 3 commits July 4, 2025 00:11
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@mattisonchao mattisonchao requested a review from Copilot July 3, 2025 16:16
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for a closable health server to ensure gRPC shutdown does not hang due to open health watch streams.

  • Introduces ClosableHealthServer in common/rpc and replaces all uses of health.NewServer with rpc.NewClosableHealthServer
  • Updates Server.Close to call the new Close() on the health server
  • Adds TestNewServerClosableWithHealthWatch to verify that health watch streams close cleanly

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
server/server_test.go Added TestNewServerClosableWithHealthWatch but missing import for fmt
server/server.go Switched from *health.Server to rpc.HealthServer and updated Close logic
server/internal_rpc_server_test.go Updated tests to use rpc.NewClosableHealthServer
server/internal_rpc_server.go Changed parameter and field type to rpc.HealthServer
server/assignment_dispatcher_test.go Updated dispatcher tests to use rpc.NewClosableHealthServer
server/assignment_dispatcher.go Updated dispatcher to accept rpc.HealthServer and use it in standalone dispatcher
common/rpc/closable_health_server.go Implemented ClosableHealthServer with Close(), context cancellation, and watch handling
Comments suppressed due to low confidence (1)

server/server_test.go:67

  • The fmt package is used here but not imported; add import "fmt" at the top of this test file.
	client, closer, err := clientPool.GetHealthRpc(fmt.Sprintf("127.0.0.1:%v", server.InternalPort()))

@mattisonchao mattisonchao merged commit 6cb8907 into main Jul 4, 2025
8 checks passed
@mattisonchao mattisonchao deleted the fix.server.stuck branch July 4, 2025 06:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants