Skip to content

Small workaround to allow istiod to not use values#19449

Merged
istio-testing merged 8 commits intoistio:masterfrom
costinm:istiod-inject
Dec 10, 2019
Merged

Small workaround to allow istiod to not use values#19449
istio-testing merged 8 commits intoistio:masterfrom
costinm:istiod-inject

Conversation

@costinm
Copy link
Copy Markdown
Contributor

@costinm costinm commented Dec 6, 2019

Follow up to istiod install, this allows migrating the majority of 'values.yaml' settings from injector.

Note that istio/api#1194 will add the inject env to mesh.yaml - and after it is
merged I'll adjust this accordingly ( with the agreed name, and merging the env from mesh.yaml)
We will still need to support env overrides - istiod allows using a k8s configmap with 'vendor' specific settings, created for example when the cluster is created.

The final plan is to use only mesh.yaml and merge 'vendor/cluster' mesh.yaml with 'install' defaults and user overrides.

But on the short term - this PR allows progress on cleaning up the injection template and removing
the dep on values.yaml.

[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[*] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

@googlebot googlebot added the cla: yes Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA. label Dec 6, 2019
@istio-testing istio-testing added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Dec 6, 2019
// Migrating from values.yaml to mesh or env (for istiod - set using fromEnv instead of values)
// This allows an optional config map with vendor-specific settings/overrides - alternative is creating
// vendor-specific injection templates or a huge mess.
funcMap["value"] = func(key string, def string) string {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get it. So in the template we have something like {{ value "foo.bar" "" }} and it will lookup the environment variable foo_bar if it exists, and if not it will get .global.foo.bar?

Why not just do something like {{ valueOrDefault (getEnv "PILOT_FOO_BAR") .Values.global.foo.bar }}? its much more explicit I think

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That will make other problems below go away

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have getEnv - if I can add getEnv instead it's great - it was the first implementation :-)

But knowing people are sensitive to env variables - I tried to not expose them directly.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems more desired to just explicitly add an getEnv function rather than to create one, mask it under a name value that does it anyways but is less flexible and doesn't work in a lot of cases?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed to {{ env }}

// Special case for 'Values.global' - where most settings are
if !strings.Contains(key, ".") {
global := values["global"]
if globalM, ok := global.(map[string]interface{}); ok {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this won't work if you have a nested value like foo.bar, right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it will work if I find the code used by galley/cobra/etc - having env variables override cli and yaml is already supported somewhere.

Again - it's not intended to be the final solution, istio/api changes is the right path and happy to remove this code as soon as we get the needed changes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After rename - this is gone.

Also: the istio/api change adds 'ProxyConfig.env' - after it is merged I'll start using it directly, with same semantics.

if globalM, ok := global.(map[string]interface{}); ok {
sval := globalM[key]
if sval != nil {
if val, ok := sval.(string); ok {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why only support string?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not intended as a generic jsonpatch reinvention - just a way to not have to keep values.yaml around for injector. I'll add whatever is needed - so far I replaced only strings.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to reinvent anything with {{ valueOrDefault (getEnv "PILOT_FOO_BAR") .Values.global.foo.bar }} other than added the env function though?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well - the point is to stop using .Values.

But it doesn't matter - since the api will be 'env' map in proxy config, that's the way to go.

This PR is just a stop gap until api is agreed.

@costinm costinm requested a review from a team as a code owner December 9, 2019 21:11
// Allows the template to use env variables from istiod.
// Istiod will use a custom template, without 'values.yaml', and the pod will have
// an optional 'vendor' configmap where additional settings can be defined.
funcMap["env"] = func(key string, def string) string {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default is not used?

Also nit, this can probably be inline in the funcMap declaration?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is used now.

There is nothing else in the funcMap declaration that inlines stuff - consistency.

conIDresourceNamePrefix, discReq.Node.Id, err)
return err
}
sdsServiceLog.Infoa("Pushed secret for ", discReq)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to make these in another PR if you want only my approval to merge this

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. Fixed.

@costinm
Copy link
Copy Markdown
Contributor Author

costinm commented Dec 10, 2019

PTAL

// Istiod will use a custom template, without 'values.yaml', and the pod will have
// an optional 'vendor' configmap where additional settings can be defined.
funcMap["env"] = func(key string, def string) string {
val := os.Getenv(key)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we ever want to set an env var to "" explicitly? if so, we should use lookup env which can distinguish unset and ""

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we find a use case - we could. But by that time - the mesh API will be in place, and we can replace this code with the equivalent ( we'll still need injector to process the 'vendor' envs, but
will be placed on overrides of mesh.proxyConfig.env )

@howardjohn
Copy link
Copy Markdown
Member

/retest

@costinm
Copy link
Copy Markdown
Contributor Author

costinm commented Dec 10, 2019

/test unit-tests_istio

@istio-testing istio-testing merged commit 96cff59 into istio:master Dec 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants