Skip to content

feat: Ignore ready and stats listener metrics in shutdown manager calculation#7985

Merged
zirain merged 8 commits intoenvoyproxy:mainfrom
zirain:fix-7947
Jan 28, 2026
Merged

feat: Ignore ready and stats listener metrics in shutdown manager calculation#7985
zirain merged 8 commits intoenvoyproxy:mainfrom
zirain:fix-7947

Conversation

@zirain
Copy link
Copy Markdown
Member

@zirain zirain commented Jan 19, 2026

fixes: #7947

use listener.(.*).downstream_cx_active stats instead of server.total_connections which contains ready and stats listener.

@zirain zirain requested a review from a team as a code owner January 19, 2026 04:37
@netlify
Copy link
Copy Markdown

netlify bot commented Jan 19, 2026

Deploy Preview for cerulean-figolla-1f9435 ready!

Name Link
🔨 Latest commit 2d9ad56
🔍 Latest deploy log https://app.netlify.com/projects/cerulean-figolla-1f9435/deploys/697a320ef3b895000889422c
😎 Deploy Preview https://deploy-preview-7985--cerulean-figolla-1f9435.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@zirain zirain changed the title feat: Ignore ready and stats listener metrics in shutdown manager cal… feat: Ignore ready and stats listener metrics in shutdown manager calculation Jan 19, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Jan 19, 2026

Codecov Report

❌ Patch coverage is 58.13953% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.90%. Comparing base (15b62c0) to head (2d9ad56).
⚠️ Report is 26 commits behind head on main.

Files with missing lines Patch % Lines
internal/cmd/envoy/shutdown_manager.go 58.13% 15 Missing and 3 partials ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #7985   +/-   ##
=======================================
  Coverage   73.89%   73.90%           
=======================================
  Files         237      237           
  Lines       35831    35847   +16     
=======================================
+ Hits        26478    26491   +13     
  Misses       7496     7496           
- Partials     1857     1860    +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

} else {
defer resp.Body.Close()
}
defer func() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can there be a case where resp is nil

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, resp shouldn't be nil when there's no error.

@zirain zirain added this to the v1.7.0-rc.1 Release milestone Jan 28, 2026
arkodg
arkodg previously approved these changes Jan 28, 2026
Copy link
Copy Markdown
Contributor

@arkodg arkodg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @zirain, can we add an e2e or confirm with a manual test that this works, i.e. envoy exits faster than default drain duration

@arkodg arkodg requested review from a team January 28, 2026 03:54
@zirain
Copy link
Copy Markdown
Member Author

zirain commented Jan 28, 2026

thanks @zirain, can we add an e2e or confirm with a manual test that this works, i.e. envoy exits faster than default drain duration

2026-01-28T04:31:27.402Z        INFO    shutdown-manager        envoy/shutdown_manager.go:93    received shutdown ready request
2026-01-28T04:31:27.500Z        INFO    shutdown-manager        envoy/shutdown_manager.go:130   initiating drain with 10 second minimum drain period and 60 second timeout
2026-01-28T04:31:27.503Z        INFO    shutdown-manager        envoy/shutdown_manager.go:244   total downstream connections: 0
2026-01-28T04:31:28.504Z        INFO    shutdown-manager        envoy/shutdown_manager.go:244   total downstream connections: 0
2026-01-28T04:31:29.511Z        INFO    shutdown-manager        envoy/shutdown_manager.go:244   total downstream connections: 0
2026-01-28T04:31:30.515Z        INFO    shutdown-manager        envoy/shutdown_manager.go:244   total downstream connections: 0
2026-01-28T04:31:31.524Z        INFO    shutdown-manager        envoy/shutdown_manager.go:244   total downstream connections: 0
2026-01-28T04:31:32.526Z        INFO    shutdown-manager        envoy/shutdown_manager.go:244   total downstream connections: 0
2026-01-28T04:31:33.529Z        INFO    shutdown-manager        envoy/shutdown_manager.go:244   total downstream connections: 0
2026-01-28T04:31:34.530Z        INFO    shutdown-manager        envoy/shutdown_manager.go:244   total downstream connections: 0
2026-01-28T04:31:35.533Z        INFO    shutdown-manager        envoy/shutdown_manager.go:244   total downstream connections: 0
2026-01-28T04:31:36.538Z        INFO    shutdown-manager        envoy/shutdown_manager.go:244   total downstream connections: 0
2026-01-28T04:31:37.541Z        INFO    shutdown-manager        envoy/shutdown_manager.go:244   total downstream connections: 0
2026-01-28T04:31:37.541Z        INFO    shutdown-manager        envoy/shutdown_manager.go:149   minimum drain period reached; will exit when total connections reaches 0
2026-01-28T04:31:37.541Z        INFO    shutdown-manager        envoy/shutdown_manager.go:157   drain sequence completed
2026-01-28T04:31:37.553Z        INFO    shutdown-manager        envoy/shutdown_manager.go:67    received terminated
2026-01-28T04:31:38.427Z        INFO    shutdown-manager        envoy/shutdown_manager.go:112   shutdown readiness detected

arkodg
arkodg previously approved these changes Jan 28, 2026
zhaohuabing
zhaohuabing previously approved these changes Jan 28, 2026
Copy link
Copy Markdown
Member

@zhaohuabing zhaohuabing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking nit: it could help later readers if we can explain each pattern in the skipConnectionRE.

@zirain zirain dismissed stale reviews from zhaohuabing and arkodg via 48d51c8 January 28, 2026 05:09
@zirain
Copy link
Copy Markdown
Member Author

zirain commented Jan 28, 2026

wait #8090

…culation

Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: zirain <zirain2009@gmail.com>
@cnvergence
Copy link
Copy Markdown
Member

/retest

@zirain zirain merged commit 762ea33 into envoyproxy:main Jan 28, 2026
57 of 59 checks passed
@zirain zirain deleted the fix-7947 branch January 28, 2026 22:21
SadmiB pushed a commit to SadmiB/gateway that referenced this pull request Jan 30, 2026
…culation (envoyproxy#7985)

* feat: Ignore ready and stats listener metrics in shutdown manager calculation

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* refactor

Signed-off-by: zirain <zirain2009@gmail.com>

* remove USE_SERVER_CONNECTIONS

Signed-off-by: zirain <zirain2009@gmail.com>

* address review comment

Signed-off-by: zirain <zirain2009@gmail.com>

* display the real value

Signed-off-by: zirain <zirain2009@gmail.com>

* comment for worker thread

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: Sadmi Bouhafs <sadmibouhafs@gmail.com>
zirain added a commit to zirain/gateway that referenced this pull request Feb 9, 2026
…culation (envoyproxy#7985)

* feat: Ignore ready and stats listener metrics in shutdown manager calculation

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* refactor

Signed-off-by: zirain <zirain2009@gmail.com>

* remove USE_SERVER_CONNECTIONS

Signed-off-by: zirain <zirain2009@gmail.com>

* address review comment

Signed-off-by: zirain <zirain2009@gmail.com>

* display the real value

Signed-off-by: zirain <zirain2009@gmail.com>

* comment for worker thread

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>
zirain added a commit to zirain/gateway that referenced this pull request Feb 9, 2026
…culation (envoyproxy#7985)

* feat: Ignore ready and stats listener metrics in shutdown manager calculation

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* refactor

Signed-off-by: zirain <zirain2009@gmail.com>

* remove USE_SERVER_CONNECTIONS

Signed-off-by: zirain <zirain2009@gmail.com>

* address review comment

Signed-off-by: zirain <zirain2009@gmail.com>

* display the real value

Signed-off-by: zirain <zirain2009@gmail.com>

* comment for worker thread

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>
zirain added a commit that referenced this pull request Feb 11, 2026
* feat: Ignore ready and stats listener metrics in shutdown manager calculation (#7985)

* feat: Ignore ready and stats listener metrics in shutdown manager calculation

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* refactor

Signed-off-by: zirain <zirain2009@gmail.com>

* remove USE_SERVER_CONNECTIONS

Signed-off-by: zirain <zirain2009@gmail.com>

* address review comment

Signed-off-by: zirain <zirain2009@gmail.com>

* display the real value

Signed-off-by: zirain <zirain2009@gmail.com>

* comment for worker thread

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>

* fix: custom response should be put at the first of the filter chain (#8061)

* fix: custom response should be put before oauth2

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* move the custom response filter to first

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* add release note

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

---------

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix: route idle timeout (#8058)

* fix: route idle timeout

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* address comments

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* add test

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

---------

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix: remove global logger in message package (#8131)

* fix: remove global logger in message package

Signed-off-by: zirain <zirain2009@gmail.com>

* fix gen

Signed-off-by: zirain <zirain2009@gmail.com>

* fix: controller cache-sync readiness check (#7430)

Signed-off-by: zirain <zirain2009@gmail.com>

* release notes for v1.5.9 (#8219)

* release notes for v1.5.9

Signed-off-by: zirain <zirain2009@gmail.com>

* update

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>

* update VERSION

Signed-off-by: zirain <zirain2009@gmail.com>

* update release notes

Signed-off-by: zirain <zirain2009@gmail.com>

* update

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Co-authored-by: Isaac Wilson <10012479+jukie@users.noreply.github.com>
zirain added a commit that referenced this pull request Feb 11, 2026
* fix(status): align BackendTLSPolicy ResolvedRefs reason with Gateway API (#7793)

* fix(status): align BackendTLSPolicy ResolvedRefs reason with Gateway API

Signed-off-by: Aditya7880900936 <adityasanskarsrivastav788@gmail.com>

* fix(gatewayapi): use accurate error for missing CA bundle in BackendTLSPolicy

Signed-off-by: Aditya7880900936 <adityasanskarsrivastav788@gmail.com>

* gatewayapi: fix BackendTLSPolicy status reasons for invalid CA refs

Signed-off-by: Aditya7880900936 <adityasanskarsrivastav788@gmail.com>

* Update internal/gatewayapi/backendtlspolicy.go

Co-authored-by: Arko Dasgupta <arkodg@users.noreply.github.com>
Signed-off-by: Aditya Sanskar Srivastav <161202916+Aditya7880900936@users.noreply.github.com>

* gatewayapi: align BackendTLSPolicy invalid CA status and formatting

Signed-off-by: Aditya7880900936 <adityasanskarsrivastav788@gmail.com>

* gatewayapi: align BackendTLSPolicy invalid CA error message with validation output

Signed-off-by: Aditya7880900936 <adityasanskarsrivastav788@gmail.com>

* testdata: regenerate BackendTLSPolicy invalid CA output

Signed-off-by: Aditya7880900936 <adityasanskarsrivastav788@gmail.com>

* fix(gatewayapi): keep Accepted reason as NoValidCACertificate for invalid CA ref kind

Signed-off-by: Aditya7880900936 <adityasanskarsrivastav788@gmail.com>

* chore(gatewayapi): fix import grouping in BackendTLSPolicy

Signed-off-by: Aditya7880900936 <adityasanskarsrivastav788@gmail.com>

---------

Signed-off-by: Aditya7880900936 <adityasanskarsrivastav788@gmail.com>
Signed-off-by: Aditya Sanskar Srivastav <161202916+Aditya7880900936@users.noreply.github.com>
Co-authored-by: Arko Dasgupta <arkodg@users.noreply.github.com>

* feat: Ignore ready and stats listener metrics in shutdown manager calculation (#7985)

* feat: Ignore ready and stats listener metrics in shutdown manager calculation

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* refactor

Signed-off-by: zirain <zirain2009@gmail.com>

* remove USE_SERVER_CONNECTIONS

Signed-off-by: zirain <zirain2009@gmail.com>

* address review comment

Signed-off-by: zirain <zirain2009@gmail.com>

* display the real value

Signed-off-by: zirain <zirain2009@gmail.com>

* comment for worker thread

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>

* fix: custom response should be put at the first of the filter chain (#8061)

* fix: custom response should be put before oauth2

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* move the custom response filter to first

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* add release note

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

---------

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix: route idle timeout (#8058)

* fix: route idle timeout

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* address comments

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* add test

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

---------

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix: remove global logger in message package (#8131)

* fix: remove global logger in message package

Signed-off-by: zirain <zirain2009@gmail.com>

* fix: TCPRoute mTLS didn't work (#8152)

* fix: remove auto HTTP config on TCP cluster

Signed-off-by: zirain <zirain2009@gmail.com>

* fix lint

Signed-off-by: zirain <zirain2009@gmail.com>

* add e2e

Signed-off-by: zirain <zirain2009@gmail.com>

* fix e2e

Signed-off-by: zirain <zirain2009@gmail.com>

* fix comment

Signed-off-by: zirain <zirain2009@gmail.com>

* fix

Signed-off-by: zirain <zirain2009@gmail.com>

* fix resource name

Signed-off-by: zirain <zirain2009@gmail.com>

* address Arko's comment

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>

* fix: continue processing the remaining xDS with invalid EnvoyPatchPolicies (#8153)

continue processing the remaining xDS with invalid EnvoyPatchPolicies

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix gen

Signed-off-by: zirain <zirain2009@gmail.com>

* fix gen

Signed-off-by: zirain <zirain2009@gmail.com>

* fix: controller cache-sync readiness check (#7430)

Signed-off-by: zirain <zirain2009@gmail.com>

* fix gen

Signed-off-by: zirain <zirain2009@gmail.com>

* release notes for v1.6.4 (#8221)

* release notes for v1.6.4

Signed-off-by: zirain <zirain2009@gmail.com>

* update

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: zirain <zirain2009@gmail.com>

* update VERSION

Signed-off-by: zirain <zirain2009@gmail.com>

* update release notes

Signed-off-by: zirain <zirain2009@gmail.com>

* update

Signed-off-by: zirain <zirain2009@gmail.com>

---------

Signed-off-by: Aditya7880900936 <adityasanskarsrivastav788@gmail.com>
Signed-off-by: Aditya Sanskar Srivastav <161202916+Aditya7880900936@users.noreply.github.com>
Signed-off-by: zirain <zirain2009@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Co-authored-by: Aditya Sanskar Srivastav <161202916+Aditya7880900936@users.noreply.github.com>
Co-authored-by: Arko Dasgupta <arkodg@users.noreply.github.com>
Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Co-authored-by: Isaac Wilson <10012479+jukie@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ignore ready and stats listener metrics in shutdown manager calculation

6 participants