Skip to content

Listener: implement internal listener#16763

Closed
lambdai wants to merge 8 commits intoenvoyproxy:mainfrom
lambdai:addinternallistener
Closed

Listener: implement internal listener#16763
lambdai wants to merge 8 commits intoenvoyproxy:mainfrom
lambdai:addinternallistener

Conversation

@lambdai
Copy link
Copy Markdown
Contributor

@lambdai lambdai commented Jun 2, 2021

Commit Message:
Allow adding internal address listener to the envoy.
This PR covers the listener CRUD but the dispatch is not yet providing the functionality
to create a connection to the internal listener.

The end2end connection establishment can be found in #16095
The internal connection saves ~3% cpu usage vs a tcp listener on loopback device at 1Gbps transmit rate.

Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:

Runtime guard: envoy.reloadable_features.internal_address
[Optional Fixes #Issue]
[Optional Deprecated:]
[Optional API Considerations:]

lambdai added 4 commits June 1, 2021 10:57
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
@lambdai
Copy link
Copy Markdown
Contributor Author

lambdai commented Jun 2, 2021

Perhaps we need two reviewers: @antoniovicente can you review the dispatcher part? Last time we chatted that there is per worker component to create the connection, which doesn't have to be the dispatcher. However, I feel the dispatcher is the best fit.

Signed-off-by: Yuchen Dai <silentdai@gmail.com>
socklen_t error_size = sizeof(error);
RELEASE_ASSERT(socket_->getSocketOption(SOL_SOCKET, SO_ERROR, &error, &error_size).rc_ == 0,
"");
if (Runtime::runtimeFeatureEnabled("envoy.reloadable_features.internal_address") &&
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nicer if we could have internal connections act more like standard connections. Having to special case internal connections in Network::ConnectionImpl and other places makes thinking about these connection types much harder. Could we try to make internal connections a more well-behaved and self-contained extension?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no fundamental known problem to make internal connection acts as connection.
Can I list it as TODO? This PR is not trying to restrict the changes at InternalListener add/update/delete, this place is proof of concept that a connection could be instantiated.
Most importantly, when the runtime feature is disabled it's clear that the normal connection doesn't have behavior change

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Take line 876 below for example. We have an ASSERT that seems to be only relevant if connection type is not internal. Would it be possible to remove the conditionals in this file that depend on connection type == internal?

config_ = &config;
}

void ActiveTcpSocket::onTimeout() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is very large and difficult to review. Should you try to do a pure refactor of this class in a separate PR before extending its functionality?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry! This part doesn't bring logic change and the existing test cases cover each line of the function.

I am open to create a dedicated PR but I also want some opinions from @htuch and @mattklein123 to see if that PR is greatly reduce the load of review.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see references to internal connection type in some of the modified files, so at least some parts of this PR is not a pure refactor.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think it would make review much more easier and faster if you can split this into two PRs:

  1. The pure refactor, which hopefully we can process without too much bike shed and eliminate a large amount of the churn.
  2. The new internal connection aspects. This is where the meat of the PR is, and where we should be able to spend the bulk of the review time.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@antoniovicente @htuch Is #16947 a smooth refactor to you?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm looking through it right now. It is not a trivial refactor. I see several class renames and new uses of the curiously recurring template pattern for not immediately clear reasons.

Signed-off-by: Yuchen Dai <silentdai@gmail.com>
@lambdai
Copy link
Copy Markdown
Contributor Author

lambdai commented Jun 9, 2021

/wait

Signed-off-by: Yuchen Dai <silentdai@gmail.com>
}
}

void ActiveInternalListener::newConnection(Network::ConnectionSocketPtr&& socket,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

feel like this whole method can be moved to base class since it looks same with the ActiveTcpListener one.

@lambdai
Copy link
Copy Markdown
Contributor Author

lambdai commented Jun 11, 2021

Creating new for refactor
/wait

@github-actions
Copy link
Copy Markdown

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@github-actions github-actions bot added the stale stalebot believes this issue/PR has not been touched recently label Jul 12, 2021
@github-actions
Copy link
Copy Markdown

This pull request has been automatically closed because it has not had activity in the last 37 days. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@github-actions github-actions bot closed this Jul 19, 2021
zuercher pushed a commit that referenced this pull request Sep 2, 2021
Client connection can be created with least changes using underlying user space io socket per comment #16763 


Risk Level: low
Testing: unit tests
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: n/a

Signed-off-by: Yuchen Dai <silentdai@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale stalebot believes this issue/PR has not been touched recently waiting

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants