Skip to content

Istiod agent#18314

Merged
istio-testing merged 45 commits intoistio:masterfrom
costinm:istiod-agent
Nov 8, 2019
Merged

Istiod agent#18314
istio-testing merged 45 commits intoistio:masterfrom
costinm:istiod-agent

Conversation

@costinm
Copy link
Copy Markdown
Contributor

@costinm costinm commented Oct 25, 2019

Initial merging of (minimal) CA functionality from Citadel into istiod - just the GRPC cert signing and self-signed cert.

Initial merging of SDS grpc into pilot-agent - just minimal stuff to call istiod or google ca to get certs.

Tested manually - still working on running the e2e tests against this.

@costinm costinm requested a review from a team October 25, 2019 00:50
@costinm costinm requested review from a team as code owners October 25, 2019 00:50
@istio-testing istio-testing added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Oct 25, 2019
@googlebot googlebot added the cla: yes Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA. label Oct 25, 2019
@costinm
Copy link
Copy Markdown
Contributor Author

costinm commented Oct 25, 2019

Note - this is built on top of #18132, waiting for that to merge - the last commits are relevant to CA.

@istio-testing istio-testing added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Oct 25, 2019
@costinm
Copy link
Copy Markdown
Contributor Author

costinm commented Nov 1, 2019

/test integ-galley-k8s-tests_istio
/test integ-security-k8s-tests_istio

@costinm
Copy link
Copy Markdown
Contributor Author

costinm commented Nov 1, 2019

/test e2e-simpleTests_istio
/test unit-tests_istio

@costinm
Copy link
Copy Markdown
Contributor Author

costinm commented Nov 1, 2019

/test integ-security-k8s-tests_istio

@costinm
Copy link
Copy Markdown
Contributor Author

costinm commented Nov 4, 2019

/test pilot-multicluster-e2e_istio

@costinm
Copy link
Copy Markdown
Contributor Author

costinm commented Nov 4, 2019

/test unit-tests_istio
/test pilot-multicluster-e2e_istio

@costinm
Copy link
Copy Markdown
Contributor Author

costinm commented Nov 6, 2019

/test integ-security-k8s-tests_istio


// Location of K8S CA root.
k8sCAPath = "./var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Jimmy, I think we should reuse 15020 if it's already used?

@lei-tang
Copy link
Copy Markdown
Contributor

lei-tang commented Nov 7, 2019

The PR description says the PR is only tested manually. For each flow related to this PR, can you provide detailed intructions on how to test this PR manually?

conIDresourceNamePrefix := sdsLogPrefix(conID, resourceName)
if !s.skipToken {
if s.localJWT {
// Running in-process, no need to pass the token from envoy to agent as in-context - use the file
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am uncertain why the localJWT option is needed. Without the localJWT option, Envoy reads the token from a file and passes it to istio agent in the context. Withou localJWT option, istio agent reads the token from a file.
If the localJWT option is introduced to improve the performance, as the token still needs to be read, there is no performance gain.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to simplify the code - there is no need for Pilot to generate a complicated config, with the file path - in each cluster/listener - when the agent has access to the file as well.

The option is only used short term, until node-agent is still in use - once it's gone there will be no passing of tokens from envoy to SDS.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a fun fact - if the file is not found, envoy is crashing hard.

Not passing the file from envoy is not only simpler - it also allows us the option to use Envoy GRPC library for envoy to local SDS communication, instead of using 2 grpc stacks at the same time.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Be a good citizen -- file an issue about the crash?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think John did already, he found it while testing istiod.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide more details about "using 2 grpc stacks at the same time"?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

@costinm costinm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re. testing: this depends on a separate PR in installer, with a new injection template and istiod updates.

The important part - that the new SDS doesn't affect existing workloads - is covered by the existing test suites.

@costinm
Copy link
Copy Markdown
Contributor Author

costinm commented Nov 7, 2019

/test integ-security-k8s-tests_istio

@howardjohn
Copy link
Copy Markdown
Member

/retest

@istio-testing istio-testing merged commit b9defec into istio:master Nov 8, 2019
@istio-testing
Copy link
Copy Markdown
Collaborator

@costinm: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
integ-istioio-k8s-tests_istio ab6a17d link /test integ-istioio-k8s-tests_istio
Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

howardjohn added a commit that referenced this pull request Nov 8, 2019
sdake pushed a commit to sdake/istio that referenced this pull request Dec 1, 2019
* Incremental changes:

- remove the SDS startup in pilot - the feedback is to start with SDS in
agent, pilot will only sign certs.
- cleanup a bit the startup
- use port 8080 for http - istioctl expects it.
- add the original Dockerfile - while it is getting integrated into the
istio build system ( can be useful after as well, it's very easy)

* Add a cloudbuilder for istiod, add missing files

* More cleanup, remove controversial dockerfile

* Use upstream galley server, reuse kube

* Make format

* Fix lint

* Add missing trust domain

* Post merge fixes

* Lint

* Fix nil check for interface

* Lint

* Initial SDS agent and support

* Improved startup config, use injection

* Tested SDS with Istiod and google plugin

* Format and lint

* More lint fighting, cleanup

* Change the template to use K8S-signed cert with local SDS

* More fixes, better debug

* Lint and fixes

* More lint

* Simpler/safer code. Istiod will not support citadel agent

* Restore previous code to disable sds

* Another attempt to format code and fight linter

* Indent and fix sds default

* Fix another crazy startup issue.

The watcher is not only watching - but also kicks the first start.
This code needs to be mostly replaced once SDS and istiod are enabled.

Also add a simple way to patch galley options for fine tunning. Like
values.yaml subset needed by injection, it's part of the transition to
mesh.yaml

* Add explicit option to use local JWT

* format

* Review feedback, renames and more comments

* Test failure

* Signature change in security

* Fixes for crash and canary mode

* Remove dep on bootstrap
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants