Allow user to locally obfuscate secret in a keystore#5687
Allow user to locally obfuscate secret in a keystore#5687tsg merged 2 commits intoelastic:masterfrom ph:feature/keystore
Conversation
There was a problem hiding this comment.
Self note: Add test for bad or corrupt keystore.
libbeat/paths/paths.go
Outdated
There was a problem hiding this comment.
self note: remove this line, thank you vim.
andrewkroh
left a comment
There was a problem hiding this comment.
Nice job figuring out the AES-GCM stuff.
libbeat/cmd/keystore.go
Outdated
There was a problem hiding this comment.
s/GenKeystoreCmd/genKeystoreCmd/ (probably you haven't linted yet since it's not marked for review and I'm jumping the gun)
There was a problem hiding this comment.
I have the linter on as I code, It used to be called GenKeystoreCmd, but I've renamed it since it doesn't need to be exported in any way. Not sure why my linter didn't catch it.
I run:
Enabled Linters: ['gofmt', 'golint', 'go vet']
libbeat/cmd/keystore.go
Outdated
There was a problem hiding this comment.
How about Failed to create the secret: no key provided?
libbeat/keystore/file_keystore.go
Outdated
There was a problem hiding this comment.
Since these comments are rendered in our godoc pages can you please add a period to the end of the sentence. https://github.com/golang/go/wiki/CodeReviewComments#comment-sentences
libbeat/keystore/file_keystore.go
Outdated
There was a problem hiding this comment.
I would only register a defer like this after you have "os.Create()ed" or "os.OpenFile()ed". But based on the code you have I think you could add a single os.Remove() call to the error handling for the failed os.Rename().
libbeat/keystore/file_keystore.go
Outdated
There was a problem hiding this comment.
You should do this check before adding the above defer or else the file you're asking them to delete will be cleaned up already.
There was a problem hiding this comment.
I've removed the defer, so this will also fix this.
libbeat/keystore/secure_string.go
Outdated
There was a problem hiding this comment.
What about GoString()? Do we need to implement that too to cover %#v or does String() cover that case too? https://golang.org/pkg/fmt/#GoStringer
There was a problem hiding this comment.
I believe we also need to cover that, I will add a test case for it.
libbeat/keystore/file_keystore.go
Outdated
There was a problem hiding this comment.
os.Stat is going to return a nil error if the file exists. So should the logic be if err == nil && !override?
There was a problem hiding this comment.
I've changed the logic, it should if the File exist and we don't want to override.
if os.IsExist(err) && !override {
return ErrAlreadyExists
}There was a problem hiding this comment.
Using os.IsExists() is for checking the error and a nil error will be returned if the file exists. See the implementation for a better understanding.
libbeat/keystore/keystore.go
Outdated
There was a problem hiding this comment.
Based on this description maybe the method should be IsPersisted(). And then the implementation would take into account the dirty flag.
libbeat/keystore/keystore.go
Outdated
libbeat/cmd/instance/beat.go
Outdated
There was a problem hiding this comment.
Use a lower cased error message. https://github.com/golang/go/wiki/CodeReviewComments#error-strings
NOTICE.txt
Outdated
There was a problem hiding this comment.
Could you update the revision of the dependencies in an separate PR? This would make sure the version change does not break other things and it would make this PR much smaller.
There was a problem hiding this comment.
Agree, I've marked as a TODO in the PR description.
|
@ruflin @andrewkroh @urso I have updated this PR with the following:
If you review #5735 I will rebase this PR this will make it easier to review. Thanks |
|
@joshbressers This is the PR for the keystore implementation in filebeat, its closer to the kibana implementation than the LS version. |
|
jenkins test this please |
- Added: golang.org/x/crypto/pbkdf2 (Key derivation function) - Added: golang.org/x/crypto/ssh/terminal (Request a password securely on the terminal) - Updated: golang.org/x/sys/unix (required by terminal) - Updated: golang.org/x/sys/windows (required by terminal) Required by: #5687
|
Rebased with master, the vendored files are now gone! :) |
ruflin
left a comment
There was a problem hiding this comment.
Skimmed through the code more on a high level. Looks good to me so far, I think it will be also helpful to play around with this feature. And we should do some follow up PR's which also touch other projects but would be overkill to be added to this PR.
libbeat/cmd/instance/beat.go
Outdated
There was a problem hiding this comment.
I suggest you check the error before overwriting the config?
libbeat/cmd/keystore.go
Outdated
There was a problem hiding this comment.
I initially wanted to comment on why you exit here instead of returning an error and handling it up stream. But I saw that this kind of a common pattern now in cmd package. @exekias Do we need to exit here or could we also return and error? In the past we tried to reduce the places of Exit to 1 place to make testing easier.
There was a problem hiding this comment.
Most of theses cmd are tested using the python framework, no? In this case exiting early with os.Exit(1) make sense? There is an [helper in cobra](https://github.com/spf13/cobra/blob/e5f66de850af3302fbe378c8acded2b0fa55472c/cobra/cmd/helpers.go#L63 to do that.
There was a problem hiding this comment.
We should some helpers like Fatalf and Fatal that log and do os.Exit(1).
There was a problem hiding this comment.
btw. the log.Fatal(f) methods do print the message + do os.Exit(1). It's common practice to use this one in main function on simple tools.
There was a problem hiding this comment.
+1 for returning errors and having top-level do the exit handling :)
As cobra requires a Run function one can wrap it like this:
func runWith(fn func (args []string) error) func(*cobra.Command, []string) {
return func(_ *cobra.Command, args []string) {
if err := fn(args); err != nil {
fmt.Error(err)
os.Exit(1)
}
}
}
...
Run: runWith(func(args []string) error {
return errors.New("TODO")
})
There was a problem hiding this comment.
yeah, and doing this through the logger has the benefit that the logger can flush and sync before exiting.
There was a problem hiding this comment.
@urso @andrewkroh I can make the changes on the keystore cmd, we can create another issue to refactor the other command to have the same flow or execution, sound like plan?
There was a problem hiding this comment.
Actually, lets do a PR with the runWith change and slowly refactor commands, I think its better to split concerns in multiple PR.
There was a problem hiding this comment.
Sorry, I missed the notification here, but I totally agree with @urso, those run wrappers are +:100:
libbeat/common/bytes_test.go
Outdated
There was a problem hiding this comment.
using assert.NotEqual(t, string(v1[:]), string(v2[:])) would probably give a nicer error message.
libbeat/common/terminal/terminal.go
Outdated
There was a problem hiding this comment.
A bit scary that we now have interactive terminal support :-D
There was a problem hiding this comment.
Yes, but all the command can be used without requiring interactive calls, so we are still friendly for orchestration.
libbeat/common/terminal/terminal.go
Outdated
There was a problem hiding this comment.
I assume @andrewkroh is not going to like these newlines even thought it's not after a { ;-) Often the err check is directly after without a newline.
libbeat/keystore/file_keystore.go
Outdated
There was a problem hiding this comment.
As there are potentially coming more key stores in the future, I'm thinking if we should put each in it's own package. keystore/file/... for this one, so this would be just New. BTW that is refactoring that could also be done in a second step.
There was a problem hiding this comment.
Agree with that, since the pluggable part of the keystore is not complete, I think we should do it when we add more.
libbeat/keystore/file_keystore.go
Outdated
There was a problem hiding this comment.
We should really abstract out on how we write files from the filebeat registry and winlobeat registry: https://github.com/elastic/beats/blob/master/filebeat/registrar/registrar.go#L230 There are strange issues that can happen on shutdown and I remember @andrewkroh fixed on of those in winlogbeat. Having the logic on one place would make sure fixes are applied everywhere.
Also see the safeFileRotate helper: https://github.com/elastic/beats/blob/master/filebeat/registrar/registrar.go#L251
@ph Could we create a follow up meta issue to track such things we should cleanup after getting this PR in. Happy to help with abstracting out this logic.
libbeat/cmd/keystore.go
Outdated
There was a problem hiding this comment.
another option to decouple the cobra.Command from the function:
func genCreateKeystoreCmd(name, version string) *cobra.Command {
var force bool
cmd := &cobra.Command{
Use: ...
Short: ...
Run: func(cmd *cobra.Command, args []string) {
if err := runCreateKeystore(name, version, force); err != nil {
fmt.Error(err)
}
}
}
cmd.Flags().BoolVar(&force, "force", false, "...")
return cmd
}
Common options can be added to the parent-command by using PersistentFlags().
libbeat/common/terminal/terminal.go
Outdated
There was a problem hiding this comment.
Wonder if we want a 'secure' Input. One that doesn't echo.
There was a problem hiding this comment.
Yes its provided by "golang.org/x/crypto/ssh/terminal" which take care of the different implementation
|
There is one that I use in the add key is provided by a ssh related lib
On Wed, Nov 29, 2017 at 5:41 PM Steffen Siering ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In libbeat/common/terminal/terminal.go
<#5687 (comment)>:
> @@ -0,0 +1,57 @@
+package terminal
+
+import (
+ "bufio"
+ "fmt"
+ "os"
+ "strings"
+)
+
+// ReadInput Capture user input for a question
+func ReadInput() (string, error) {
Wonder if we want a 'secure' Input. One that doesn't echo.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#5687 (review)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAACgIJDaO2KRdKueKtkI5DkZBsvY2mUks5s7d2egaJpZM4Qnuc3>
.
--
ph
|
libbeat/common/config.go
Outdated
There was a problem hiding this comment.
Why do we need to export this? common.Config type fully wraps go-ucfg, such that no other component has to deal with passing the options.
There was a problem hiding this comment.
I've removed the export of this field.
|
A minor style one on newline usage :) :
|
|
@tsg I am taking a look, I did most of my tests using filebeat, I will revisit path expansion I have probably miss something when we define a new resolver on the configuration. |
|
@tsg The string replacement is working on my side when you are defining it in the configuration yml, I will do a bit more investigation. I also added system env test for filebeat, I will push the changes soon. I will test metricbeat/packetbeat maybe something is different for them. Concerning the |
|
For the -E, seems we don't configure the keystore resolver at the right time. |
|
@tsg The problem is In short, ucfg, try to make the configuration concrete when we merge. I think we need to implement a PartialyConcrete concept in our configuration, still trying to figure out how to do it. |
|
I stumbled upon the cyclic issue with autodiscover too, ended up namespacing everything with |
I am looking into that, going in the rabbit hole :) |
libbeat/cfgfile/cfgfile.go
Outdated
There was a problem hiding this comment.
exported function GetKeystorePath should have comment or be unexported
|
@tsg @exekias I went deep into the rabbit hole on this one but I don't have the fix for it, to not waste any more time I need to sync with @urso on that because I think this will need internal changes on ucfg to correctly fixes our uses cases. I've experimented with a lot of different ways to fix it, and it ends all the same. I had the prior discussion with @urso concerning improvements that we wanted to make on the config, I think we should take this one our priority. Concerning the cyclic reference, I have applied a fix to detect when a cyclic reference happens an error should be returned, the problem is theses errors are not bubbled up in the stack they are just ignored and this happen before reaching the stack. Also, I was expecting my changes to break the test suite which was not the case. Like @exekias said I think the following would solve the problem, we load or merges configuration we should be able (via a flag?) to retain Reference as a type of the configuration. This would allow us to do work with partially resolved options:
The configuration is a chicken/egg problem, and without supporting partially resolved configuration, I don't think we can support Also, another thing I've tried in my local experimentations is to make sure |
|
After discussing with @monicasarbu we think we can move it forward without fixing the cyclic reference in that PR and do it in another PR. @tsg I will double check everything with my lastest changes and ping you for another real world testing. |
|
@tsg Ready for another round of testing! |
libbeat/cmd/instance/beat.go
Outdated
There was a problem hiding this comment.
I wonder if path.data (typically /usr/share/beatname/data is the right place for the keystore. Alternatively we could consider path.config (typically /etc/beatname)?
We should probably follow what ES/KB did, do you know?
Also, I guess we'd want to check the permissions on the file like we do with the YAML config files.
There was a problem hiding this comment.
Right, /etc/beatname make more sense, I haven't checked ES/KB I will verify that and make the change if needed.
We do check for permission https://github.com/elastic/beats/pull/5687/files#diff-33a534765f797732e272dc741db414f4R233 if StrictPerm(0600) is True.
|
@ph did you do any tests of the commands on Windows? Just very slightly worried about the |
|
@tsg Concerning windows, I've tested it early and it was OK, but I will do another complete run just to be on the safe side. |
|
@tsg small issue on windows powershell, will get a fix for that. Good point, I am still assuming more compatibility than I should coming from the jvm. |
|
@tsg Windows is all good and happy now with the exact same behavior as unix. |
|
@tsg I've addressed your concerns, adding log debug when successfully resolving keys from the keystore and where the keystore is loaded. Also I've changed the default path of the keystore to |
|
jenkins test this please |
|
@ph The last Jenkins run is almost green, but there's a failure on Windows that looks related to the PR: I triggered another Travis run on Filebeat as well. |
tsg
left a comment
There was a problem hiding this comment.
LGTM, if the test failure is solved.
This PR allow users to define sensitive information into an obfuscated data store on disk instead of having them defined in plaintext in the yaml configuration. This add a few users facing commands: beat keystore create beat keystore add output.elasticsearch.password beat keystore remove output.elasticsearch.password beat keystore list The current implementation doesn't allow user to configure the secret with a custom password, this will come in future improvements of this feature.
|
@tsg I've pushed a new commit for the test on windows, I will monitor the greeness of it. The Appveyor tests are also failing for winlogbeat, but this should not be related to anything this PR touch,. |
|
|
||
| // Stretch the user provided key | ||
| password, _ := k.password.Get() | ||
| passwordBytes := pbkdf2.Key(password, salt, iterationsCount, keyLength, sha512.New) |
There was a problem hiding this comment.
Consider replacing pbkdf2 with a never key derivation function like scrypt or even Argon2. Argon2 was very recently merged into golang.org/x/crypto/ .
Some resources about the topic:
https://www.linkedin.com/pulse/top-password-hashing-schemes-employ-today-chintan-jain-mba-cissp
https://download.libsodium.org/doc/password_hashing/
https://core.trac.wordpress.org/ticket/39499
https://gitlab.com/cryptsetup/cryptsetup/issues/119
In addition I would recommend to store the key derivation function and the iteration along side with the VERSION|SALT|IV|PAYLOAD.
If you decide to change the iteration count or the key derivation function you don't need to increase the file format version.
This is kind of similar to the https://secure.php.net/manual/en/function.password-hash.php. In your case the the key would be left out.
With this implementation it's possible to increase the key derivation parameters to prevent faster CPU/GPU's. To achieve this the maintainer needs to alter the key derivation parameters from time to time and if a change is detected from the key derivation parameters stored in the storage file the encrypted payload needs to be decrypted and encrypted with the new parameters.
Similar to https://secure.php.net/manual/en/function.password-needs-rehash.php
To reduce code duplication I would extract the passwordBytes generation passwordBytes := pbkdf2.Key(password, salt, iterationsCount, keyLength, sha512.New) to it own private function.
Btw: The rest of the encryption part looks good to me.
|
Good suggestion I will do a follow up on that.
On Fri, Jan 5, 2018 at 6:29 PM Dominic ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In libbeat/keystore/file_keystore.go
<#5687 (comment)>:
> + // randomly generate the salt and the initialization vector, this information will be saved
+ // on disk in the file as part of the header
+ iv, err := common.RandomBytes(iVLength)
+
+ if err != nil {
+ return nil, err
+ }
+
+ salt, err := common.RandomBytes(saltLength)
+ if err != nil {
+ return nil, err
+ }
+
+ // Stretch the user provided key
+ password, _ := k.password.Get()
+ passwordBytes := pbkdf2.Key(password, salt, iterationsCount, keyLength, sha512.New)
Consider replacing pbkdf2 with a never key derivation function like scrypt
or even Argon2. Argon2 was very recently merged into golang.org/x/crypto/
.
Some resources about the topic:
https://www.linkedin.com/pulse/top-password-hashing-schemes-employ-today-chintan-jain-mba-cissp
https://download.libsodium.org/doc/password_hashing/
https://core.trac.wordpress.org/ticket/39499
https://gitlab.com/cryptsetup/cryptsetup/issues/119
In addition I would recommend to store the key derivation function and the
iteration along side with the VERSION|SALT|IV|PAYLOAD.
If you decide to change the iteration count or the key derivation function
you don't need to increase the file format version.
This is kind of similar to the
https://secure.php.net/manual/en/function.password-hash.php. In your case
the the key would be left out.
With this implementation it's possible to increase the key derivation
parameters to prevent faster CPU/GPU's. To achieve this the maintainer
needs to alter the key derivation parameters from time to time and if a
change is detected from the key derivation parameters stored in the storage
file the encrypted payload needs to be decrypted and encrypted with the new
parameters.
Similar to
https://secure.php.net/manual/en/function.password-needs-rehash.php
To reduce code duplication I would extract the passwordBytes generation passwordBytes
:= pbkdf2.Key(password, salt, iterationsCount, keyLength, sha512.New) to
it own private function.
Btw: The rest of the encryption part looks good to me.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#5687 (review)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAACgIZqPS5LnJi_-FzjyI2BdzmSYzG7ks5tHrBHgaJpZM4Qnuc3>
.
--
ph
|
This PR allow users to define sensitive information into an obfuscated data store on disk instead of having them defined in plaintext in the yaml configuration.
This add a few users facing commands:
The current implementation doesn't allow user to configure the secret with a custom password, this will come in future improvements of this feature.
How to use it
You can reference keys from the keystore using the same syntax that we use for the environment variables.
When we unpack the configuration we will resolve the variables using the following priority:
TODO before review
[ ] raise an error if we try override a config defined in the yaml file. (Another PR)This is the initial version, I don't expect it to be pluggable with every system, but some of the things are in place to make it pluggable.(ie: interface/factory)
closes: #3982