feat: introduce secret rewrite merge operation#4894
feat: introduce secret rewrite merge operation#4894Skarlso merged 17 commits intoexternal-secrets:mainfrom
Conversation
|
why the hell did helm-release run here? 🤔 |
I didn't do nothing, I swear! 🥲 A few quesitons:
|
a4614b8 to
ef98618
Compare
Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com>
…writes Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com>
Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com>
Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com>
Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com>
Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com>
…rations Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com>
ef98618 to
d6979f3
Compare
… merge Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com>
15c387d to
fbdc2b1
Compare
|
@riccardomc Yes please, if you could take care of the sonar issue that would be fantastic. :) 🙇 |
Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com>
|
@Skarlso sure, here you go! 😉 |
|
Thanks! :) |
Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com>
a127ba2 to
4945ea8
Compare
Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com>
Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com>
|
|
This looks incredible! |
|
Nice. :) This has been released with 0.18.1. :) Enjoy! |
* initial implementation of rewrite merge Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> * introduce stable secrets merging order to ensure predictable key overwrites Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> * dataFrom.rewrite.merge: implement key priority Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> * introduce conflictPolicy & strategy, refactor Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> * polish code, improve tests, add combined rewrite tests Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> * add documentation Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> * make Error default conflict policy for ExternalSecretRewriteMerge operations Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> * update CRDs after setting Error as default conflictPolicy for rewrite merge Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> * reduce cognitive complexity of RewriteMap method Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> * improve error messages by adding failed rewrite operation type Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> * use maps.Copy() in RewriteMerge to propagate input map Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> * introduce check if key in priority list exists in input Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> --------- Signed-off-by: Riccardo M. Cefala <riccardo.c@miro.com> Co-authored-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com> Signed-off-by: asrormirzoev <asrormirzoev@yandex-team.ru>
|
@riccardomc Just wanted to say I finally had the chance to start working with this today and it's working perfectly! |
|
Quick update Seems like this implementation is missing the "if they exist" part, as if I try to create an externalSecret with a "priority" configuration, but the key specified does not exist, it will result in an error. For example, for this: If "my-app/secrets" does not exist, will result in:
While I CAN see value in this causing an error if a secret is missing in some use-cases (for example if there's a secret you want to mandate), I think the "this key is prioritized if it exists" approach makes more sense, at least as the default. @riccardomc May I please request a follow up to either make prioritized secrets optional, or if you see fit, add a "priorityStrategy" field for either "Error" or "Ignore" 👉👈 |
|
Hey, thanks! I am very happy you're using it and thanks for reaching out, much appreciated. The main reason I decided to keep this behavior is to minimize surprises for end users: if we just ignore the non existing remote secret, then the final content in the Secret might be different than expected without very little possibility of noticing. I thought that this could lead to very unpleasant circumstances: it's quite easy to fail to override credentials that might cause significant issues at the application level. Imagine failing to override Production connection string because the Staging remote secret has a typo. Maybe the Would you mind opening a separate Issue for this as this is now merged? |
|
@riccardomc sure, will do, thanks! |



Problem Statement
This pull request implements the suggestion in #2987 which allows to merge secret values from different secrets obtained from a provider through a
dataFrom.findin one singleSecretaccording to configurable priority, merge strategy and conflict policy.Related Issue
Implements #2987
Proposed Changes
The implementation extends the
ExternalSecretDataFromRemoteRefAPI by introducing a newExternalSecretRewriteMergemethod to instruct the controller on how to perform merge operations on secret key/value pairs as part ofExternalSecretRewrite.The implementation follows this detailed suggestion comment for the most part, with some interpretations and choices.
Found secret keys are sorted in alphabetical order before being processed. This guarantees a stable and predictable conflict resolution order. Keys specified in the
prioritylist are processed after all the others, guaranteeing that the last key on theprioritylist has precedence over all the others.Documentation and extensive unit tests are provided to further illustrate the implementation details.
Additional Notes
main. So I would like to get the ball rolling nonetheless and see if they pass in CI (update: passed...). Here's a snippet of the locally failing tests:Checklist
git commit --signoffmake testmake reviewable