@mattfarina did the update with the master and corrected the changes, I would like feedback.

@hunsche = lifesaver! Thank you sir.

Thanks for keeping the original pr text, there are some issues still to do, as described there. @hunsche Thoughts around the open issues?

Hello @vladfr, I would like to discuss these topics.

About these two items:

• currently, files are parsed first, then dirs; if a file overlaps somehow, the dir will overwrite it. What if we parse files last, so you can potentially overwrite an unwanted file in a dir? This one is easy to do
• all external files are added to the root of the chart. This means you can overwrite any chart file, including any template. Should we add these files to a special _external dir? Chart authors will need to append it to every lookup: {{ (.Files.Glob "_external/certs/") }}

I think it is the responsibility of the graphic developer to understand the overlap of the files, but if it is necessary to generate a warning or validation it is relatively simple.

About this item:

• if a path is not found, is unreadable, or if a glob yields no files, the helm command currently fails. Should we continue the install with a warning?

I really like the idea of ​​not failing, but a warning may be necessary.

Hello,
I'm currently directly looking into this PR as it solves one of our biggest problems.
I tried the PR locally and it works for our use-case; I'm looking forward to get this merged.
Please let me know if i can help.

my feedback to the current questions:

currently, files are parsed first, then dirs; if a file overlaps somehow, the dir will overwrite it. What if we parse files last, so you can potentially overwrite an unwanted file in a dir? This one is easy to do

No strong feeling here, it might make sense to allow files to be more specific that globs;
I plan to use this with grafana dashboards, here could see some value by being able to do:
--include-dir gf-dashboards/*.json --include-file patched-dashboard.json

if a path is not found, is unreadable, or if a glob yields no files, the helm command currently fails. Should we continue the install with a warning?

Initially i was thinking that failing is the better solution but on a second thought it might be better to not fail; If we think again about the grafana dashboards example, I also would like to deploy grafana with my CICD even if no dashboard matches the glob.

all external files are added to the root of the chart. This means you can overwrite any chart file, including any template. Should we add these files to a special _external dir? Chart authors will need to append it to every lookup: {{ (.Files.Glob "_external/certs/") }}

No strong feeling here either, I would argue that its up to the user to avoid the conflicts and not rely on an "_external" structure

Hi, any updates on this please. I am eagerly waiting for this PR to be merged so that I can use it in my implementation.

would be nice to have this soon

From my view at the moment the uncertainty is around some of the open topics and a decision needs to be done;
or was it already decided to go forward with the current status?

@hunsche, @mattfarina any advice on who can take a decision around those points?
I'm new to the helm contribution model, I checked the documentation but it's still no clear who can or will review PRs.

I had a quick look and code-wise any of the three mentioned points seem to be easy to solve.

For what it's worth, I migrated a chart to use the forked branch and have this working correctly

any advice on who can take a decision around those points?

Have a look at the Contributing Guide, specifically the section on PR lifecycles. Anyone from the OWNERS file can review and approve this PR.

At the moment, we've been focused and more concerned about the Helm 2 deprecation plan that was scheduled for last week. Now that milestone is behind us, we can re-focus on reviewing PRs like this one.

Thanks for your patience.

@bacongobbler thanks for the clarification, I checked the contributing guidelines, but it was not clear for me who can review.

Let me know if i can help on this one.

Any update on this please. Could someone please review and merge this PR so that we can use it.

From my view at the moment the uncertainty is around some of the open topics and a decision needs to be done;
or was it already decided to go forward with the current status?

@hunsche, @mattfarina any advice on who can take a decision around those points?
I'm new to the helm contribution model, I checked the documentation but it's still no clear who can or will review PRs.

I had a quick look and code-wise any of the three mentioned points seem to be easy to solve.

For what it's worth, I migrated a chart to use the forked branch and have this working correctly

I believe we can follow, with these points as is, and collect feedback for improvements in new versions.

I just spent 6 hours trying various workarounds, and then I found this which would solve all my problems 🥰
I'm really looking forward to this getting merged! And it seems ready to go too? Great work @hunsche 😉
Ready to hit the merge button @bacongobbler ? 🚀

How do rollbacks work in this scenario?

Say I do the following:

1. helm install --include-dir=foo/bar myrelease somechart (Creates myrelease.v1)
2. helm upgrade myrelease (Creates myrelease.v2)
3. helm rollback myrelease (Rolls back to myrelease.v1)`

In step 3, Helm may attempt to re-render resources. Does it still have access to the content of --include-dir? Where are those files located in the release record?

Thank you @hunsche for this feature contribution.

I've gone through the discussion in #3276 :)

I typically handle this use case by

1. Having tooling which creates secrets/configmaps
2. Have a way to reference them in my values.yaml

Would the included files be treated as values which happen to be rendered from Go templates, conceptually ?

The reason I bring this up in the PR is that:

• if we are treating the files in --include-file as a special kind of "values", we should treat them that way and not worry about re-rendering on rollback.
• If we treat them as an extension of the content in /templates , then re-rendering on rollback is a possibility we need to consider.

I'm sorry that we were not able to get to this prior to preparing to release v3.5.0. We are moving into the release candidate phase so this will need to be looked at for 3.6.

I was just researching how to do this with helm - these flags look just what I need

Is there an ETA when this will be done? It is painful currently having to rely on symlinks instead of this feature.

I'm striving for the next version.

Hey, just wanted to mention that our team is really looking forward for this feature.
We currently have some ugly patches in our code and this could solve them beautifully.
Hope this gets released soon :)

Just want to update that we talked about this feature at the Helm dev call, and decided that we need to write documentation and tests for the expected behaviour of these flags :)

Usually helm charts are self documented (as in what can be config) by looking at the values.yaml file. Would merging this make this the first time we break this rule (do we even have this rule?)?

Anyway, updating Chart.yaml schema and perform file/flag validation would make my concern above less of an issue. Could even make this play well with rancher or some other software that install chart by UI.

I still think some use cases could be resolved without this PR by using kubectl create secret. Could not cover all use cases though. Not really urgent merge IMO.

  # Create a new secret named my-secret with keys for each file in folder bar
  kubectl create secret generic my-secret --from-file=path/to/bar
  
  # Create a new secret named my-secret with specified keys instead of names on disk
  kubectl create secret generic my-secret --from-file=ssh-privatekey=path/to/id_rsa
--from-file=ssh-publickey=path/to/id_rsa.pub
  
  # Create a new secret named my-secret with key1=supersecret and key2=topsecret
  kubectl create secret generic my-secret --from-literal=key1=supersecret --from-literal=key2=topsecret
  
  # Create a new secret named my-secret using a combination of a file and a literal
  kubectl create secret generic my-secret --from-file=ssh-privatekey=path/to/id_rsa --from-literal=passphrase=topsecret
  
  # Create a new secret named my-secret from an env file
  kubectl create secret generic my-secret --from-env-file=path/to/bar.env

@duyleekun In the case of my team, your suggestion couldn't solve our use case. We have a Helm chart for the application we're developing, and we provide the app's configurations with it. Then, when you install the chart, you install with the command: helm install <release-name> <chart-name> --set env=staging, and it'll deploy the app with its staging configurations.
Because of this use case, we have the configurations duplicated - once for the local development of the app (can run without K8s or even without Docker), and the second is inside our chart's directory.

Hope it's clear :)

I think you could use --set envSecrets[0]=secret-name-for-staging, similar to the imagePullSecrets we had for ages. Just my idea.

I still think this could create a lot of potentials to chart developer, I wouldn't mind if we can make this friendlier to user along the way

@duyleekun I understand what you're suggesting, and we've thought about it for quite some time, but it's not as convenient because then we need to remember to update the configurations on each deployment, instead of just installing the chart.
This feature could really help us and I think it could help more users.

Btw, I've thought about it more, and I think this feature could be improved by a few simple additions that I'd like to discuss with you.
First of all, when installing a chart accessing external files, the current behavior is that it returns nothing. I think this is a bit unexpected, and that it should either fail or print a warning telling the user to pass the added flags.
The second suggestion is that we could also add the flags to helm package, and it'll make the packaged chart include the files that were present at the time. It'll then make the "external files" known only to the developers and the user installing the chart shouldn't even care that that chart includes files that didn't arrive from the chart's working directory.

What do you think about these additions? I'd be glad to hear your opinions on them :)

First of all, when installing a chart accessing external files, the current behavior is that it returns nothing. I think this is a bit unexpected, and that it should either fail or print a warning telling the user to pass the added flags.

I agree with your idea about fail or warn, as I noted above, it should be document officially in the schema or someplace. I'm a bit skeptical while reading README.md of any helm chart, I usually read non template file to see what the chart expect from me.

The second suggestion is that we could also add the flags to helm package, and it'll make the packaged chart include the files that were present at the time. It'll then make the "external files" known only to the developers and the user installing the chart shouldn't even care that that chart includes files that didn't arrive from the chart's working directory.

I thought this was implemented in the current release by putting files in the template folder, I could be wrong though. I guess your idea is to provide sensible default file embedded in the chart so user can replace whatever file they want later. IMO, this kind of make sense because I expect charts to be functional OOB and they usually do. I don't really have many use cases in mind yet for this suggestion yet, will certainly throw more idea later.

@duyleekun I read the code in this PR and it seems that when you run helm package, the packaged chart will have {{ .Files.Get ~/my_file }}.
I'm planning to start working on this soon (depending on my personal schedule) and hopefully I'll open a PR for these additions soon :)

Hi, any updates on this please.... looking forward to this feature...

Hi, any updates on this please.... looking forward to this feature...

I know.. I wish I could still work on this, but unfortunately I can't continue working on this soon.
You're welcome to continue my work here if you can :)

Hi, any updates on this please.... looking forward to this feature...

I know.. I wish I could still work on this, but unfortunately I can't continue working on this soon. You're welcome to continue my work here if you can :)

what else needs to be done to get this completed?

Can we get it shipped somehow? 😃 Badly needed for monorepo(s) 🙄

@technosophos
About rollback.
The end result is a yaml equal to the build if it's embedded in the values.
The implementation is not making a reference to a directory, it only allows access to a directory outside the chart.

My doubt is, why is this allowed:

helm install --values /tmp/some-external-values.yaml myrelease somechart

Where /tmp/some-external-values.yaml is

externalConf: |-
   totalExternalFile

---
apiVersion: v1
kind: ConfigMap
metadata:
   name: "RELEASE-NAME-"
date:
   external.conf: |-
{{ .Values.externalConf }}

And generate file:

---
# Source: configmap/templates/config-map.yaml
apiVersion: v1
kind: ConfigMap
metadata:
   name: "RELEASE-NAME-"
date:
   external.conf: |-
      totalExternalFile

And accessing another directory outside the chart is not allowed?

The end result will be the same format as yaml

The whole community waits for this.
Please stop blocking this functionality

Since @hunsche hasn't been able to work on this for a very long time and @itaispiegel picked it back up with #10077, I'm closing this PR in favor of that one.