Skip to content

[Galley] Adding ServiceEntry synthesis#11293

Closed
nmittler wants to merge 1 commit intoistio:release-1.1from
nmittler:galley-serviceentries
Closed

[Galley] Adding ServiceEntry synthesis#11293
nmittler wants to merge 1 commit intoistio:release-1.1from
nmittler:galley-serviceentries

Conversation

@nmittler
Copy link
Copy Markdown
Contributor

@nmittler nmittler commented Jan 26, 2019

Added a new custom projection that is subscribed to events for k8s Pods, Nodes, Services and Endpoints. These events are absorbed and do not become part of the snapshot. Instead, synthetic ServiceEntry resources are generated and become part of the snapshot.

Partially addresses #10497 and #10589

@nmittler nmittler requested review from ayj, costinm and ozevren January 26, 2019 01:14
@istio-testing istio-testing added the do-not-merge/work-in-progress Block merging of a PR because it isn't ready yet. label Jan 26, 2019
@nmittler nmittler force-pushed the galley-serviceentries branch from da57306 to c8df2ff Compare January 28, 2019 16:05
@nmittler nmittler force-pushed the galley-serviceentries branch from c8df2ff to 7ae150c Compare January 28, 2019 18:07
@nmittler
Copy link
Copy Markdown
Contributor Author

@ozevren @ayj I've moved the code reorg into a separate PR so they don't pollute this one: #11325

Let's get agreement there first ... then I'll rebase and we can review this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be nice for these constants to be accessible by Pilot so we don't need to redefine them (e.g. see here).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as discussed offline ... I'll remove the custom collection here in favor of just reusing the existing ServiceEntry collection. Pilot will know that these are special since: 1) it will have to create a client that listens on a different group (not "default") and 2) We'll add a special annotation to indicate that the ServiceEntry is synthetic.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And duplicating the constants is less of a problem than introducing a dependency.

@nmittler nmittler removed the request for review from cmluciano January 28, 2019 21:33
@nmittler nmittler added this to the 1.1 milestone Jan 28, 2019
@nmittler nmittler force-pushed the galley-serviceentries branch from 7ae150c to a8e195a Compare January 28, 2019 22:31
Copy link
Copy Markdown
Member

@sdake sdake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nmittler I understand this PR is WIP. Looks pretty good already! How do you plan to enable this for the operator? ENV variable, config flag? How do you enable this PR in your local env (e.g. if I wanted to eval the PR, how would I?)

Cheers
-steve

@nmittler nmittler force-pushed the galley-serviceentries branch from a8e195a to f7d4dd7 Compare January 29, 2019 17:51
@nmittler nmittler changed the title [WIP] [Galley] Adding ServiceEntry synthesis. [WIP] [Galley] Adding ServiceEntry synthesis Jan 29, 2019
@nmittler nmittler requested review from Nino-K and andraxylia January 29, 2019 17:52
@nmittler
Copy link
Copy Markdown
Contributor Author

@sdake

@nmittler I understand this PR is WIP. Looks pretty good already! How do you plan to enable this for the operator? ENV variable, config flag? How do you enable this PR in your local env (e.g. if I wanted to eval the PR, how would I?)

Currently, the new flow will be enabled at the source end. I've previously introduced a new flag that filters creation of sources for certain types. By default, this new functionality is "off", since the default filter excludes Pods, Nodes, Endpoints, and Services.

@ozevren @ayj anything to add here?

@nmittler nmittler force-pushed the galley-serviceentries branch 4 times, most recently from d7eba74 to 5808d0a Compare January 30, 2019 00:40
@rshriram
Copy link
Copy Markdown
Member

is this okay to merge? Would like some ack from galley team

@ozevren
Copy link
Copy Markdown
Contributor

ozevren commented Feb 20, 2019

Looked at this before the last update with benchmarks, lgtm. I'll take a look again for the recent changes. As for getting this in, I don't think this is part of the PR short list @duderino is keeping for 1.1 though.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit: if you use t.Run(...) equivalent of b here, then you don't need to reset timer. It should also account memory allocations only within the context of the lambda.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just not sure what the test name should be in that case. I'd rather err on the side of a clear test name here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make these data driven? It looks like there is an opportunity to set the right data model here for adding more variations of the test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can revisit once we have a second data set.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a problem with your checkin in particular, but do the ns and serviceAccount need to be escaped?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure.

@quanjielin @wattli may have more

@nmittler
Copy link
Copy Markdown
Contributor Author

@ozevren PTAL ... added one more commit, which adds an integration-level benchmark

@nmittler nmittler force-pushed the galley-serviceentries branch from 93593bc to 7344080 Compare February 20, 2019 21:09
@nmittler
Copy link
Copy Markdown
Contributor Author

/test e2e-mixer-no_auth

Copy link
Copy Markdown
Contributor Author

@nmittler nmittler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ozevren PTAL

@nmittler nmittler force-pushed the galley-serviceentries branch from d594205 to 8ef20f3 Compare February 21, 2019 20:25
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just simply set the debug log level as part of init and not worry about resetting it? Would make the test code cleaner.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered that, but I'd rather not interfere with other tests.

@nmittler nmittler force-pushed the galley-serviceentries branch 2 times, most recently from 0fa500e to 367cac5 Compare February 22, 2019 19:21
@nmittler
Copy link
Copy Markdown
Contributor Author

/test istio-pilot-e2e-envoyv2-v1alpha3

Added a new custom projection that is subscribed to events for k8s Pods, Nodes, Services and Endpoints. These events are absorbed and do not become part of the snapshot. Instead, synthetic ServiceEntry resources are generated and become part of the snapshot.

Partially addresses istio#10497 and istio#10589
@istio-testing
Copy link
Copy Markdown
Collaborator

@nmittler: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
prow/e2e-simpleTests-cni.sh 99ffdbf link /test e2e-simpleTests-cni
prow/istio-integ-k8s-tests.sh 99ffdbf link /test istio-integ-k8s-tests
prow/e2e_pilotv2_auth_sds.sh 99ffdbf link /test istio_auth_sds_e2e
Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@andraxylia
Copy link
Copy Markdown
Contributor

The new file synthetic/serviceEntry contains in fact k8s nodes, pods, services and it is under networking/v1alpha3.

I was expecting that directory to contain only Istio APIs and serviceEntry.yaml to contain maybe some examples of generated ServiceEntry.

The KRMs should be somewhere else, maybe in the file-system adapter?

@nmittler
Copy link
Copy Markdown
Contributor Author

nmittler commented Mar 2, 2019

@andraxylia

The new file synthetic/serviceEntry contains in fact k8s nodes, pods, services and it is under networking/v1alpha3.

I was expecting that directory to contain only Istio APIs and serviceEntry.yaml to contain maybe some examples of generated ServiceEntry.

It doesn't really matter much where these files go. I chose to put it under v1alpha3 because it generates a v1alpha3 proto. AFAIK there's not really a pattern established where input and output are from different packages.

The KRMs should be somewhere else, maybe in the file-system adapter?

Again, I don't entirely agree .... and it doesn't really matter much.

@andraxylia
Copy link
Copy Markdown
Contributor

It may not matter, but it is confusing, so please rename the file from serviceEntry.yaml to something else like krm.yaml (Kubernetes Resource Model). The directory networking/v1alpha3 should also contain examples for user-defined ServiceEntries, those belong in a file called serviceEntry.yaml. Please add tests for user-defined serviceEntries for external services (it can be separate PR).

@nmittler
Copy link
Copy Markdown
Contributor Author

Closing this in favor of #12409 (against the master branch)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.