WiP: Intelligent listener by lambdai · Pull Request #9773 · envoyproxy/envoy

lambdai · 2020-01-22T10:49:09Z

Description:
Intelligent listener update. Design doc: https://docs.google.com/document/d/1fjud3xSNRxxEwAWR1COsDnViB4FtTKTFgfQVW0IPU0c/edit#

The major risk originates from the fact that many references have subtle life time change.
[x] Happy path review
[ ] Structure review
[ ] readability, risk mitigation
[ ] tests
[ ] how smart the LDS could be

Risk Level: Mid, lifetime issue
Testing: TBD

Very early version: tons of tests are broken

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

lambdai · 2020-01-22T18:24:47Z

Master thread happy code path
old onListenerWarmed, new onIntelligentListenerWarmed
old addListenerToWorker+drainListener, new addIntelligentListenerToWorker+drainFilterChains

mattklein123

Thanks for working on this. I'm starting first with an interface review and then we can go from there. Thank you!

/wait

include/envoy/network/connection_handler.h

include/envoy/server/worker.h

stale · 2020-02-03T18:58:26Z

This pull request has been automatically marked as stale because it has not had activity in the last 7 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

stale · 2020-02-10T19:28:27Z

This pull request has been automatically closed because it has not had activity in the last 14 days. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

lambdai · 2020-02-24T22:46:47Z

@mattklein123 Thank you for the patience and I am back on this stuff. Could you reopen this PR?

The update:

merged the two flow as required.
rewrite removeUntrackedFilterChains and related TODOs: listener manager provides the filter chains collection to remove
add WIP integration tests: No matter the implementation details are, the goal is to allow connections survive the lds update.

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

mattklein123 · 2020-03-13T17:27:32Z

@lambdai I know you are still working on this, but please start to think about how to split this up into multiple PRs. Thank you!

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

lambdai · 2020-03-13T23:02:48Z

@mattklein123 Could you add this PR to your review queue?

mattklein123 · 2020-03-15T20:30:43Z

@lambdai is this still WIP? Per prior comment this PR is too big. Can you figure out how to split it up please? Thank you.

/wait

lambdai · 2020-03-21T07:28:02Z

Sorry I spent huge time on the another task.

I am splitting this PR into

introducing the factory context impl, which listener filter factory context and network factory context would have separate life time.
New interface to worker and listener manager so that ConnectionHandler could support network filter chain update.
The smart decision at ListenerManagerImpl which is powered by 1 and 2

stale · 2020-03-28T08:07:49Z

This pull request has been automatically marked as stale because it has not had activity in the last 7 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

stale · 2020-04-04T10:54:22Z

This pull request has been automatically closed because it has not had activity in the last 14 days. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

lambdai added 12 commits January 8, 2020 19:59

scaffold of listener inplace update

0eef61c

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

listener_impl: assume new filter chain creator

fdf07aa

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

new creator

8a102ca

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

signature: shared filter chain on build

1f69081

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

intelli listener: connection_handler

266eda8

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

intelli listener: worker

e322c66

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

intelli listener: filter chain manager

5fa27e8

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

add defer delete task impl

629f9d2

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

add prototype of removeUntrackedFilterChains series

68ce67c

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

impl tcp listener removeUntrackedFilterChains

58ec1a4

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

build listener

4a32a60

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

build envoy

5705c39

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

lambdai requested a review from mattklein123 January 22, 2020 10:49

mattklein123 self-assigned this Jan 22, 2020

mattklein123 requested changes Jan 24, 2020

View reviewed changes

include/envoy/network/connection_handler.h Outdated Show resolved Hide resolved

include/envoy/network/connection_handler.h Outdated Show resolved Hide resolved

include/envoy/server/worker.h Outdated Show resolved Hide resolved

include/envoy/server/worker.h Show resolved Hide resolved

repokitteh-read-only bot added the waiting label Jan 24, 2020

stale bot added the stale stalebot believes this issue/PR has not been touched recently label Feb 3, 2020

stale bot closed this Feb 10, 2020

lambdai added 3 commits February 21, 2020 00:20

unify add intelligent listener

f0a711d

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

untrack filter chains

3ffac8f

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

integration test

f24e6c8

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

mattklein123 reopened this Feb 24, 2020

stale bot removed the stale stalebot believes this issue/PR has not been touched recently label Feb 24, 2020

repokitteh-read-only bot removed the waiting label Feb 24, 2020

lambdai added 2 commits February 25, 2020 19:42

fill diffFilterChain and fix crashes

c95dd82

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

improve ListenerImpl::supportUpdateFilterChain

e2e8011

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

lambdai added 2 commits March 3, 2020 01:06

comments

eb5109f

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

fix and add filter chain manager test

0a93bb2

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

repokitteh-read-only bot removed the waiting label Mar 4, 2020

lambdai added 5 commits March 4, 2020 04:11

try removing draindecision from listenerimpl

4d5a142

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

drain manager for draining filter chain group

41da518

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

add conn handler test on remove filter chain

e24a6ed

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

add TcpListenerInplaceUpdate test

cd29d43

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

fix existing listener_manager_impl_test

c63a1c7

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

mattklein123 added the waiting label Mar 8, 2020

lambdai added 2 commits March 11, 2020 23:02

add test: stop and update inplace listener

3423463

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

add remove test

7ed5ce2

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

repokitteh-read-only bot removed the waiting label Mar 12, 2020

mattklein123 added the waiting label Mar 12, 2020

inteli listener drainage, manager - worker

5ecb652

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

repokitteh-read-only bot removed the waiting label Mar 12, 2020

mattklein123 added the waiting label Mar 13, 2020

restrict LdsInplaceUpdate integration test running on ipv4

538601b

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

repokitteh-read-only bot removed the waiting label Mar 13, 2020

renaming

7caa11f

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

repokitteh-read-only bot added the waiting label Mar 15, 2020

lambdai mentioned this pull request Mar 23, 2020

listener: split listenerfilterfactorycontext and networkfilterfactorycontext #10491

Merged

stale bot added the stale stalebot believes this issue/PR has not been touched recently label Mar 28, 2020

stale bot closed this Apr 4, 2020

lambdai mentioned this pull request Apr 6, 2020

listener: in place filter chain update #10662

Merged

Conversation

lambdai commented Jan 22, 2020

Uh oh!

lambdai commented Jan 22, 2020

Uh oh!

mattklein123 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stale bot commented Feb 3, 2020

Uh oh!

stale bot commented Feb 10, 2020

Uh oh!

lambdai commented Feb 24, 2020

Uh oh!

mattklein123 commented Mar 13, 2020

Uh oh!

lambdai commented Mar 13, 2020

Uh oh!

mattklein123 commented Mar 15, 2020

Uh oh!

lambdai commented Mar 21, 2020

Uh oh!

stale bot commented Mar 28, 2020

Uh oh!

stale bot commented Apr 4, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants