Skip to content

feat: add support for pushing whole Secret resources to a secret store#2288

Closed
mhannache-equativ wants to merge 1 commit intoexternal-secrets:mainfrom
mhannache-equativ:hashicorp-vault-push-whole-secret
Closed

feat: add support for pushing whole Secret resources to a secret store#2288
mhannache-equativ wants to merge 1 commit intoexternal-secrets:mainfrom
mhannache-equativ:hashicorp-vault-push-whole-secret

Conversation

@mhannache-equativ
Copy link
Copy Markdown

Problem Statement

It is not currently possible to use a PushSecret to push a whole Secret resource to a secret store. Since the secretKey field is mandatory, only a subset of the secret can be pushed.

Related Issue

Related to #2166

Proposed Changes

This MR introduce the possibility to not provide a secretKey, in this case the Secret resource's data field is converted to a JSON string and pushed to the secret store.

Checklist

  • I have read the contribution guidelines
  • All commits are signed with git commit --signoff
  • My changes have reasonable test coverage
  • All tests pass with make test
  • I ensured my PR is ready for review with make reviewable

@mhannache-equativ mhannache-equativ requested a review from a team as a code owner May 3, 2023 14:11
@mhannache-equativ mhannache-equativ requested review from sebagomez and removed request for a team May 3, 2023 14:11
Copy link
Copy Markdown
Contributor

@paul-the-alien paul-the-alien bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greetings!
Thank you for contributing to this project!
If this is your first time contributing, please make
sure to read the Developer and Contributing Process guides.
Please also mind and follow our Code of Conduct.

Useful commands:

  • make fmt: Formats the code
  • make check-diff: Ensures the branch is clean
  • make reviewable: Ensures a PR is ready for review

@mhannache-equativ mhannache-equativ changed the title feat: add support for pushing whole secret to hashicorp vault feat: add support for pushing whole Secret resource to as secret store May 3, 2023
@mhannache-equativ mhannache-equativ changed the title feat: add support for pushing whole Secret resource to as secret store feat: add support for pushing whole Secret resources to as secret store May 3, 2023
@mhannache-equativ mhannache-equativ changed the title feat: add support for pushing whole Secret resources to as secret store feat: add support for pushing whole Secret resources to a secret store May 3, 2023
@mhannache-equativ mhannache-equativ force-pushed the hashicorp-vault-push-whole-secret branch from bb7d1fa to 62de73a Compare May 4, 2023 08:30
@gusfcarvalho
Copy link
Copy Markdown
Member

@moolen this can be something to consider (besides having a PushSecret dataFrom) 🤔

@mhannache-equativ mhannache-equativ force-pushed the hashicorp-vault-push-whole-secret branch from 62de73a to 1312f1e Compare May 23, 2023 12:27
@sebagomez
Copy link
Copy Markdown
Contributor

IMO I'd prefer keeping the secret-key mandatory and having the dataFrom implemented for this scenario

@mhannache-equativ
Copy link
Copy Markdown
Author

IMO I'd prefer keeping the secret-key mandatory and having the dataFrom implemented for this scenario

I'd be willing to implement it, could you describe what kind of schema you have in mind?

@sebagomez
Copy link
Copy Markdown
Contributor

Right now you have to write something like this in your PushSecret

  data:
    - match:
        secretKey: key
        remoteRef:
          remoteKey: my-pushed-secret

If we add the dataFrom like we have it in our ExternalSecret we could have something like the following

  dataFrom:
  - find:
      name:
        regexp: "^secret-" # <-- this regex would be applied to every key in the source secret

So if the regexp is .* you would get every key from the source secret to be pushed.
I'm not sure about the structure though, I'm not sure if we would like to keep the find in case we find a good case for extract in the future.

What does everybody else think?

@moolen
Copy link
Copy Markdown
Member

moolen commented May 23, 2023

Regarding dataFrom: I think we need a design doc to figure out and document the missing use-cases for the PushSecret resource.

As for this PR i wouldn't mind to make it optional, that's simple enough and provides great value. But that's just my 5ct.

Let me summon @gusfcarvalho @knelasevero

@gusfcarvalho
Copy link
Copy Markdown
Member

gusfcarvalho commented May 25, 2023

I took a look at the other providers on how they implement PushSecret, my only concern would be some incompatibility that we may cause (i.e. instead of generating an actual key/Value secret for users to see, we generate a JSON that's not readable).

The two providers that we may run in this situation are AWS SM and Kubernetes (from what I've seen), and for those:

  • AWS SM wouldn't be aproblem because we upload it (as of right now) as a SecretBinary - we needed to see what would happen if we loaded as a SecretString
  • Kubernetes would not be a problem as we demand a property on the RemoteRef - meaning it would load the entire kubernetes secret into a kubernetes secret key (super weird, but wouldn't conflict with dataFrom).

I think we could have it as an addition indeed, if we are ok with the "weird kubernetes behavior" I've just described.

@mhannache-equativ mhannache-equativ force-pushed the hashicorp-vault-push-whole-secret branch from e2d79d4 to a6dac1c Compare May 26, 2023 13:20
@sonarqubecloud
Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

No Coverage information No Coverage information
0.0% 0.0% Duplication

@moolen moolen added this to the Push Secret beta milestone Jun 25, 2023
@gusfcarvalho
Copy link
Copy Markdown
Member

Hi @mhannache-equativ ! Sorry for the long delay. We discussed this one on our last community meeting. We can proceed with it as proposed. Can you just bump main on this PR please? 🙈 we can also do that if you're busy.

Again ,sorry for the delay.

@gusfcarvalho gusfcarvalho self-assigned this Aug 5, 2023
@mhannache-equativ
Copy link
Copy Markdown
Author

Hi @gusfcarvalho, I just rebased on main.

@mhannache-equativ mhannache-equativ force-pushed the hashicorp-vault-push-whole-secret branch from a6dac1c to 80d858d Compare August 11, 2023 10:00
@sonarqubecloud
Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

No Coverage information No Coverage information
0.0% 0.0% Duplication

@gusfcarvalho
Copy link
Copy Markdown
Member

/ok-to-test sha=80d858d

@fallen-up
Copy link
Copy Markdown

any news?

if err != nil {
return out, fmt.Errorf("error marshaling vault secret: %w", err)
}
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of marshaling the secret data we should change the PushSecret() interface. Then the provider can decide on how to handle that data.
Reason:

  • the unneccessary marshal/unmarshal is error prone: binary data isn't well supported (needs encoding, e.g. base64) and then you can't differentiate the data: is it a string, binary-base64 encoded, a jwt 🤷 ?
  • the provider doesn't know about the ref.Match, hence can not differentiate whether this belongs to a match.seretKey or this is the whole secret.

Sorry for the late reply, we're under water with PR reviews 😞
Do you have to to finish this up? If not we can take it over

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @moolen, unfortunately I will not have time to refactor the PushSecret() interface, feel free to take over.

Good luck with the other PR reviews!

@Skarlso Skarlso self-assigned this Nov 3, 2023
Signed-off-by: Marin Hannache <mhannache@equativ.com>
@Skarlso Skarlso force-pushed the hashicorp-vault-push-whole-secret branch from 80d858d to 6318bde Compare November 3, 2023 08:07
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Nov 3, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

No Coverage information No Coverage information
0.8% 0.8% Duplication

@Skarlso
Copy link
Copy Markdown
Contributor

Skarlso commented Nov 8, 2023

Superseded by #2862.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants