Skip to content

Add pilot-agent support for configuring envoy to use the metrics service#11897

Merged
rshriram merged 1 commit intoistio:release-1.1from
joeyb:envoy-metrics-service
Mar 5, 2019
Merged

Add pilot-agent support for configuring envoy to use the metrics service#11897
rshriram merged 1 commit intoistio:release-1.1from
joeyb:envoy-metrics-service

Conversation

@joeyb
Copy link
Copy Markdown
Contributor

@joeyb joeyb commented Feb 20, 2019

This PR is currently a WIP and dependent on istio/api#803. Similar to the statsd support, this extends the proxy config to support configuration of the envoy metrics service stats sink.

Remaining work:

@istio-testing
Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: joeyb
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: geeknoid

If they are not already assigned, you can assign the PR to them by writing /assign @geeknoid in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@istio-testing
Copy link
Copy Markdown
Collaborator

Hi @joeyb. Thanks for your PR.

I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@rshriram
Copy link
Copy Markdown
Member

/ok-to-test

@istio-testing istio-testing added ok-to-test Set this label allow normal testing to take place for a PR not submitted by an Istio org member. and removed needs-ok-to-test labels Feb 20, 2019
@rshriram
Copy link
Copy Markdown
Member

you need to update the helm templates for sidecar injection

@joeyb
Copy link
Copy Markdown
Contributor Author

joeyb commented Feb 20, 2019

@rshriram - I added the helm template updates to the TODO list in the description.

Btw, it's expected that the build for this would currently fail since that new ProxyConfig field doesn't actually exist yet.

@rshriram
Copy link
Copy Markdown
Member

yes.. do it all in same PR

@joeyb
Copy link
Copy Markdown
Contributor Author

joeyb commented Feb 20, 2019

@rshriram - I fleshed out the PR. I think this covers all the needed changes to propagate the new config.

For right now, I just manually copied the api changes over. Once they actually get merged, I can properly update that vendored dep.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you combine this and the envoyStatsD into one single struct like this:

# These are statistics corresponding to the inner workings of the envoy proxies in the data plane.
# They contain detailed stat information emitted by filters in envoy as well as other subsystems in Envoy,
# gathered across multiple requests and connections. This should not be confused with Istio telemetry 
# service that generates stats (e.g., response codes, latencies, etc.) per request or connection.
# Note that in addition to the following two sinks where Envoy pushes stats, prometheus style metrics can 
# be scraped from the envoys using the admin port 127.0.0.1:15000, or via 127.0.0.1:15090/stats/prometheus
# Disabled by default
envoyInternalStats:
 statsdSink
   enabled: ..
   ...
  grpcStatsReceiverSink:
   enabled: ...

and fix up the existing statsD references as well?

The reason we need to do this is because the term metrics service is utterly generic, and ends up conflating with Istio metrics/telemetry. this will cause user confusion when they want to enable/disable pieces of Istio.

On a similar note, could you please rename the API field in istio/api to envoy_internal_stats_grpc_sink ?

cc @douglas-reid - this sound good?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rshriram yes. this sounds reasonable to me. thanks for taking on this bit of work.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rshriram - The reasoning behind that merger with the statsd config makes sense, but that would also be a breaking change for helm template users that modify those values. Is that an issue? I'm fine with making the change, just want to make sure we're not breaking any expected backwards compatibility guarantees.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm true.. in that case lets keep them as two separate pieces but add the big blob on top.and probably name the metrics service thing as envoyInternalStatsGrpcReceiver for the lack of a more uglier lengthier name..

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dont forget to update the comment above

@istio-testing istio-testing added the needs-rebase Indicates a PR needs to be rebased before being merged label Feb 22, 2019
@joeyb joeyb force-pushed the envoy-metrics-service branch from 9375ff4 to ed90243 Compare February 22, 2019 21:21
@istio-testing istio-testing removed the needs-rebase Indicates a PR needs to be rebased before being merged label Feb 22, 2019
@joeyb joeyb marked this pull request as ready for review February 22, 2019 21:28
@joeyb
Copy link
Copy Markdown
Contributor Author

joeyb commented Feb 22, 2019

@rshriram - All the prep work involving istio/api should be done now and the vendored dep for it has been fully updated, so this should be ready for full review now.

@rshriram
Copy link
Copy Markdown
Member

please update that comment.. otherwise LGTM

@joeyb
Copy link
Copy Markdown
Contributor Author

joeyb commented Feb 26, 2019

@rshriram - I added some details to that comment in a previous commit, but I just pushed another commit that expands on it a bit more.

@rshriram
Copy link
Copy Markdown
Member

rebase please. I think the API update went in, in another PR by the bot.

@rshriram rshriram added this to the 1.1 milestone Feb 27, 2019
@joeyb joeyb force-pushed the envoy-metrics-service branch 3 times, most recently from b4a350b to 471e647 Compare February 27, 2019 03:33
@joeyb
Copy link
Copy Markdown
Contributor Author

joeyb commented Feb 27, 2019

@rshriram - The rebase got pretty messy, so I ended up squashing everything down to a single commit while cleaning everything up. Should be back in a good state now and ready to merge pending CI looking good.

Copy link
Copy Markdown
Member

@rshriram rshriram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please fix CI tests

@rshriram
Copy link
Copy Markdown
Member

/test istio-pilot-e2e-envoyv2-v1alpha3

@joeyb
Copy link
Copy Markdown
Contributor Author

joeyb commented Feb 28, 2019

@rshriram - afaict, those tests are all just broken. scanning through other open PRs, it looks like they are failing for everyone.

@joeyb joeyb force-pushed the envoy-metrics-service branch from 471e647 to 7942165 Compare March 4, 2019 14:46
@istio-testing
Copy link
Copy Markdown
Collaborator

@joeyb: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
prow/istio-integ-k8s-tests.sh 7942165 link /test istio-integ-k8s-tests
prow/istio-pilot-multicluster-e2e.sh 7942165 link /test istio-pilot-multicluster-e2e
Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@joeyb
Copy link
Copy Markdown
Contributor Author

joeyb commented Mar 4, 2019

@rshriram - can you take another look at this? I rebased against the latest from release-1.1 and it looks like a couple of the tests are in a better state now. It looks like the remaining failures are still unrelated to this PR.

@rshriram rshriram merged commit 305fd51 into istio:release-1.1 Mar 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Set this label allow normal testing to take place for a PR not submitted by an Istio org member.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants