Skip to content

config: ADS implementation.#1621

Closed
htuch wants to merge 11 commits intoenvoyproxy:masterfrom
htuch:ads-grpc
Closed

config: ADS implementation.#1621
htuch wants to merge 11 commits intoenvoyproxy:masterfrom
htuch:ads-grpc

Conversation

@htuch
Copy link
Copy Markdown
Member

@htuch htuch commented Sep 11, 2017

This patch introduces the AdsApi implementation, loosely based on GrpcSubscriptionImpl (there may be
a path to converge them in the future).

This patch introduces the AdsApi implementation, loosely based on GrpcSubscriptionImpl (there may be
a path to converge them in the future).
@htuch
Copy link
Copy Markdown
Member Author

htuch commented Sep 11, 2017

@mattklein123 this is still WiP. Haven't merged with envoyproxy/data-plane-api#169 yet, since it is head-of-line blocked by #1609. Also, need to add a ton more unit and integration tests, as well as address some TODOs. Just wanted to kick this into the review pipeline for early feedback.

Copy link
Copy Markdown
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at a high level looks good. Few random comments. Will look more when complete.

return cluster_load_assignment;
}

envoy::api::v2::Listener buildListener(const std::string& name, const std::string& route_config) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related to the convo we had in meeting yesterday, and to discussions with @alyssawilk, is there any way to make this more readable? By potentially having YAML templates for listeners for most things and small helper functions that are mostly abstracted? I can look at a YAML config and immediately understand what is going on. This is pretty hard to look at...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this function is particularly readable but I think the use of it is, or can be made to be.

If we

addListener("listener1, "route1")
addRoute("route1" "cluster1" "/foo");
addCluster("cluster1", Something::Mumble::HTTP1);

Then create envoy, I think it's pretty clear we have 1 listener, which maps /foo to cluster1 which speaks HTTP
That's the sort of config I'd like to see in the long run.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I agree, that's where we need to end up, with maybe the functions themselves being based on templates that are easy to understand (or at least with good comments).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have made this nicer in the commit I'm working on, here's what it will look like:

  envoy::api::v2::Cluster buildCluster(const std::string& name) {
    auto cluster = TestUtility::parseYaml<envoy::api::v2::Cluster>(R"EOF(
      connect_timeout: 5s
      type: EDS
      eds_cluster_config: { eds_config: { ads: {} } }
      lb_policy: ROUND_ROBIN
      http2_protocol_options: {}
    )EOF");
    cluster.set_name(name);
    return cluster;
  }

*/
template <class MessageType>
static inline MessageType anyConvert(const ProtobufWkt::Any& message) {
MessageType typed_message;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: I assume this needs some type of error checking?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added.

std::vector<std::string> resources_;
AdsCallbacks& callbacks_;
std::list<AdsWatchImpl*>& type_url_list_;
std::list<AdsWatchImpl*>::iterator entry_;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels a lot like LinkedObject, but it doesn't store unique_ptr internally. Would be nice to somehow reuse, but I probably not worth it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack, I know, I as looking at LinkedObject but it just wasn't the right thing for this.

try {
// To avoid O(n^2) explosion (e.g. when we have 1000s of EDS watches), we
// build a map here from resource name to resource and then walk watches_.
// TODO(htuch): Reconsider implementation data structure for watches to make lookups of resource
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, why not just store internally as map?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We expect that most watches that name resources (as opposed to open CDS/LDS watches, which just wildcard via an empty resource list) will have a single resource, but the current APIs internally allows for more. We need to assemble the subset of resources from those received and pass them to onConfigUpdate() once per watch. This means we need to do some gather operation. This could be either done by building buckets for each watch and populating them as we walk the watch list (as done today), or building buckets and populating them as we walk the resource list.

I'm not a fan of either of these. If we change the API to allow zero (wildcard) or one resource per watch, we can easily use a map. Given that's how it's actually used by the xDS implementations in Envoy today, I think I'm going to modify the ADS internal API to switch to this. Let me know if that sounds good.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 sounds good simpler is better

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After playing around with this, we still need to do the map construction. This is because we need to deliver empty updates to watches (e.g. EDS) when a resource is not present, so we basically need to walk all the watches and have a O(1) map back to the delivered resource matching the watched name. So, I'll leave it as is.

htuch added a commit to htuch/envoy that referenced this pull request Sep 13, 2017
This showed up in the ASAN/TSAN presubmits for envoyproxy#1621.

Bonus namespace cleanup in rds_subscription.cc that I hit with local clang-asan run.
htuch added a commit that referenced this pull request Sep 14, 2017
This showed up in the ASAN/TSAN presubmits for #1621.

Bonus namespace cleanup in rds_subscription.cc that I hit with local clang-asan run.
@htuch
Copy link
Copy Markdown
Member Author

htuch commented Sep 14, 2017

Closing this out for now. Going to do some rebase and major refactor stuff once nonce PRs land, not enough review history to keep.

@htuch htuch closed this Sep 14, 2017
jpsim pushed a commit that referenced this pull request Nov 28, 2022
… routes (#1621)

Risk Level: medium
hopefully addresses #1558 and #1557

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Signed-off-by: JP Simard <jp@jpsim.com>
jpsim pushed a commit that referenced this pull request Nov 28, 2022
…n to all routes (#1621)" (#1710)

* Revert "extending propercasing to legacy clusters, header sanitization to all routes (#1621)"

This reverts commit 6018cd59943da196c43ac4694ef5663e2bb45b3a.

Signed-off-by: Alan Chiu <achiu@lyft.com>

* ignore test

Signed-off-by: Alan Chiu <achiu@lyft.com>

* codespell

Signed-off-by: Alan Chiu <achiu@lyft.com>
Signed-off-by: JP Simard <jp@jpsim.com>
jpsim pushed a commit that referenced this pull request Nov 29, 2022
… routes (#1621)

Risk Level: medium
hopefully addresses #1558 and #1557

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Signed-off-by: JP Simard <jp@jpsim.com>
jpsim pushed a commit that referenced this pull request Nov 29, 2022
…n to all routes (#1621)" (#1710)

* Revert "extending propercasing to legacy clusters, header sanitization to all routes (#1621)"

This reverts commit 6018cd59943da196c43ac4694ef5663e2bb45b3a.

Signed-off-by: Alan Chiu <achiu@lyft.com>

* ignore test

Signed-off-by: Alan Chiu <achiu@lyft.com>

* codespell

Signed-off-by: Alan Chiu <achiu@lyft.com>
Signed-off-by: JP Simard <jp@jpsim.com>
mathetake pushed a commit that referenced this pull request Mar 3, 2026
…tion (#1621)

**Description**
Improve error handling by distinguishing between internal errors and
user-facing errors, returning appropriate HTTP status codes and safe
error messages to clients.

Key Changes:
1. New user_facing_errors.go with sentinel errors
2. Updated all translators to wrap errors appropriately
3. Modified request processor to detect and return user-facing errors
  4. Comprehensive test coverage across the stack

---------

Signed-off-by: yxia216 <yxia216@bloomberg.net>
Signed-off-by: Dan Sun <dsun20@bloomberg.net>
Co-authored-by: Dan Sun <dsun20@bloomberg.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants