router: vhost/route/w.cluster local filter configuration#3045
router: vhost/route/w.cluster local filter configuration#3045mattklein123 merged 14 commits intoenvoyproxy:masterfrom
Conversation
include/envoy/server/filter_config.h
Outdated
There was a problem hiding this comment.
s/RDS/Route/ ?
The route config can come from static config as well.
bazel/repository_locations.bzl
Outdated
There was a problem hiding this comment.
revert this file other than this line? those seems unrelated.
There was a problem hiding this comment.
the vscode bazel plugin seems a bit overzealous in its linting...
source/common/config/utility.h
Outdated
There was a problem hiding this comment.
I don't think you need the first template parameter, it is always Struct.
There was a problem hiding this comment.
was being consistent with the other translate function but will update!
source/common/router/config_impl.cc
Outdated
There was a problem hiding this comment.
Need to perform validation when PerFilterConfigs are loaded too.
There was a problem hiding this comment.
I don't think we can do this unfortunately. What type would we validate? (This is related to me asking very nicely for Validate() to be a method on the base proto message which is virtual and empty and can be overriden...
There was a problem hiding this comment.
I meant Validate above, but on second thought, we need a method in Factory to pre-process the route-config at loading time, not in the data path. Currently in this PR there is no way to do pre-processing of the filter config in routes. (for configs in listeners, it is done in createFilterFactoryFromProto)
There was a problem hiding this comment.
Pre-processing is being treated as an additional feature. That will need more thinking. We won't be doing that as part of this PR.
There was a problem hiding this comment.
we'll likely need to delegate this to the factory(factory) where we already ask it for the pb message. will think about this some more!
mattklein123
left a comment
There was a problem hiding this comment.
Generally looks good, thank you. Some comments to get started.
include/envoy/router/router.h
Outdated
There was a problem hiding this comment.
super nit: "the nullptr" is not typically said. Just "nullptr." Same below.
There was a problem hiding this comment.
thought it sounded funny. Thanks, the @mattklein123!
include/envoy/server/filter_config.h
Outdated
source/common/config/utility.h
Outdated
There was a problem hiding this comment.
I would just RELEASE_ASSERT() this like I recently did for the other instances of this.
There was a problem hiding this comment.
I was being consistent with the translate method above. Shall I update both?
There was a problem hiding this comment.
Yes please. I don't think it's really needed to throw an exception here.
source/common/router/BUILD
Outdated
There was a problem hiding this comment.
In general we try to not have common depend on server, though this isn't really enforced explicitly. I would add a TODO here and in code to figure out a better way to break this cycle.
There was a problem hiding this comment.
sg. i imagine this could be done with bazel visibility rules?
There was a problem hiding this comment.
I need to think about it. Will look at it as part of repo reorg work.
source/common/router/config_impl.cc
Outdated
There was a problem hiding this comment.
I don't think we can do this unfortunately. What type would we validate? (This is related to me asking very nicely for Validate() to be a method on the base proto message which is virtual and empty and can be overriden...
There was a problem hiding this comment.
s/ASSERT/EXPECT in all places for the most part.
There was a problem hiding this comment.
This has been moved and will need a master merge.
There was a problem hiding this comment.
Instead of using buffer filter, can you register a test filter for this test, so that you can correctly verify that you can override to be a different type for the filter data? If you need to you can have a proto just for testing.
There was a problem hiding this comment.
is there an existing one I can use for this or should I create one? Should I create the test proto in envoy or data plane?
There was a problem hiding this comment.
Just make one local to the test.
Signed-off-by: Chris Roche <croche@lyft.com>
Signed-off-by: Chris Roche <croche@lyft.com>
Signed-off-by: Chris Roche <croche@lyft.com>
Signed-off-by: Chris Roche <croche@lyft.com>
Signed-off-by: Chris Roche <croche@lyft.com>
Signed-off-by: Chris Roche <croche@lyft.com>
Signed-off-by: Chris Roche <croche@lyft.com>
Signed-off-by: Chris Roche <croche@lyft.com>
Signed-off-by: Chris Roche <croche@lyft.com>
b13c20c to
118c728
Compare
| * filter name configured for this virtual host. If none is present, | ||
| * nullptr is returned. | ||
| */ | ||
| virtual const Protobuf::Message* perFilterConfig(const std::string& name) const PURE; |
There was a problem hiding this comment.
This suggestion is trying to address this issue: #3008
We can do it in a separate PR. But it is so simple, you can also put in in this PR too.
In these VHost/Route/Cluster, add another function
// If perFitlerConfig exists for this filter, get its config hash value to detect if config is updated.
// return 0 if no config for the filter.
size_t perFitlerConfigHash(const std::string& name) const PURE;
There was a problem hiding this comment.
I don't quite understand how this will help you. In the filter are you going to check this on a per-route basis? What you really want is another callback which allows a filter to update global structures on a route update.
There was a problem hiding this comment.
This is to help filters need to pre-process per_route_config. per_route_config can be dynamically changed anytime. But you don't want to pre-process the config for each request. This hash will help to detect config change, and skip pre_processing step:
This is how it can be used, For each request in decodeHeader()
auto hash = route()->perFilterConfigHash("my_filter_name");
if (hash == 0) // handle not per_route_config case
obj = global_obj_map.find(hash);
if (obj == null) {
obj = pre_process_config( route()->perFilterConfig("my_filter_name") );
global_obj_map[hash] = obj;
}
This will also achieve: different routes with same config can share the same pre_processed object.
There was a problem hiding this comment.
This is not how I would recommend implementing this. (The above is not thread safe.). I have a pretty good idea of how to efficiently implement this and give you the functionality you need. Let's do this in a follow up.
There was a problem hiding this comment.
In istio mixer-client filter, the "global_map" actually is per_thread_global_map. so this approach works for us.
the filter_factory callback approach is better, the global_map can be used by all worker threads. But it is not easy for Istio to use it since its "object_map" object is way deep inside a per_thread object.
My request is: can we support both: 1) calculate the hash of per_route_config at config update time and filter can access it. 2) implements factory callback to support a config change update.
| auto& factory = Envoy::Config::Utility::getAndCheckFactory< | ||
| Server::Configuration::NamedHttpFilterConfigFactory>(name); | ||
|
|
||
| configs_[name] = Envoy::Config::Utility::translateToFactoryRouteConfig(struct_config, factory); |
There was a problem hiding this comment.
create another map
config_hash_ to store hash of config
The hash can be calculated by
https://github.com/envoyproxy/envoy/blob/master/source/common/protobuf/utility.h#L127
There was a problem hiding this comment.
Same comment as above. I don't think this is very helpful and I would rather sort out filter update callbacks as a separate feature?
Signed-off-by: Chris Roche <croche@lyft.com>
|
Ok, changes made. PTAL @mattklein123 |
mattklein123
left a comment
There was a problem hiding this comment.
LGTM, few small comments. Thank you and nice work!
| Protobuf::Map<std::string, ProtobufWkt::Struct> proto_cfgs; | ||
| proto_cfgs["unknown.filter"] = some_cfg; | ||
|
|
||
| EXPECT_THROW(PerFilterConfigs{proto_cfgs}, EnvoyException) |
There was a problem hiding this comment.
I would use EXPECT_THROW_WITH_MESSAGE here for better checking. Same elsewhere if applicable.
| << source; | ||
| } | ||
|
|
||
| std::unique_ptr<Registry::InjectFactory<Server::Configuration::NamedHttpFilterConfigFactory>> |
There was a problem hiding this comment.
nit: I think this can just be stack allocated with the test? Do you need a unique_ptr?
There was a problem hiding this comment.
It's a bit self-referential, so I was having issues creating it in the constructor, but probably can do it in the setup. Let me give that a shot.
There was a problem hiding this comment.
Ended up just splitting the filter into a nested class of the fixture
| }) << "returned message should be castable to known type"; | ||
| } | ||
|
|
||
| TEST_F(PerFilterConfigsTest, UnknownFilter) { |
There was a problem hiding this comment.
nit: This would be a better test if you had a full route config that got parsed that then resulted in the exception.
source/common/router/config_impl.cc
Outdated
| const Protobuf::Message* PerFilterConfigs::get(const std::string& name) const { | ||
| auto cfg = configs_.find(name); | ||
|
|
||
| if (cfg == configs_.end()) { |
There was a problem hiding this comment.
nit: I would just use the ternary operator here to collapse the rest of these lines into one.
|
lgtm, modulo pending comments. |
Signed-off-by: Chris Roche <croche@lyft.com>
|
@mattklein123 nits knitted! |
This patch adds support for resolving virtual host-, route-, and weighted cluster-local configuration for filters. See https://github.com/envoyproxy/data-plane-api/issues/540 for discussion. Configs are provided via RDS as opaque structs (similar to the existing LDS-provided configs), resolved to concrete message types only once, and exposed by the VirtualHost, Route and RouteEntry types. These RDS-scoped configs can be a different pb message type than the base configuration, but default to being identical. Filters can optionally access these configs to change their behavior accordingly.
I will be following up to this PR with making the buffer filter route-aware. @rshriram will also be working on the fault and lua filters to support this.
Risk Level: Low-Medium
Testing: unit
API/Docs Changes: envoyproxy/data-plane-api#605
Release Notes: (will update in this PR)