Dynamically generate template on startup#3681
Conversation
f05a8fa to
6ddff24
Compare
14492df to
62360ba
Compare
618a273 to
228ec31
Compare
Templates do not need to be checked in as they are generated before the tests and packaging. This is a prerequisit for elastic#3681 to simplify the PR.
Templates do not need to be checked in as they are generated before the tests and packaging. This is a prerequisit for #3681 to simplify the PR.
228ec31 to
d0013bb
Compare
This PR is to reduce the number of changes in elastic#3681 .
This PR is to reduce the number of changes in #3681 .
d0013bb to
e9be118
Compare
e7abc0c to
19a4e0d
Compare
heartbeat/heartbeat.full.yml
Outdated
There was a problem hiding this comment.
I assume this approach (re-using the template.path setting) just for the prototype phase or the actual plan? I'd say we probably want to rethink the configuration options.
There was a problem hiding this comment.
Agree on rethinking the config options. I added it to #3654
|
My 2c on this.
I'd say it's better to use a different name, since this is quite different conceptually. Also, I'd suggest it is a good idea to let the user provide a fully custom template file (which they can generate/edit whatever name they like). I wouldn't worry too much about backwards compatibility in this case, since I'd guess (and hope) most people didn't use a custom template path.
Yeah, configuration is tricky here, perhaps we have a global
I agree there should be a way to write the template to a file. But I guess the primary way of adjusting things will be by editing the
A binary has the disadvantage of adding to the package size. We could consider creating a |
|
Another, kind of random, thought: I'd like to make it fairly easy to change some index settings (that typically go in the template), like |
|
@tsg Agreement on the points above. I kind of start to like the idea of a beats-tools package which is mainly for sysadmins etc for the setup phase but could also be used in the future to do some "checks". I added all the points above to the meta issue to be solved #3654 I suggest moving forward with this PR as is and to further changes in follow up PR's. |
So far the template was loaded from the `beatname.template.json` files. This now uses the `fields.yml` file to dynamically generate the template based on the ES version connected to. The name of the template is set dynamically based on `template.name`. By default it has the value {beatname}.
* [x] Remove generated templates
* [x] Package fields.yml
* [x] Clean up config options as 2.x option is not needed anymore
* [x] Update docs
* [x] Update and test packaging
Further changes:
* Remove tests in libbeat to load all templates. This requires global knowledge of the other beats in libbeat which is not good.
* Create new makefile target to generate template for testing purpose.
Questions:
* Config option is called `template.path` which points to `fields.yml`. Is the name still ok?
* What is our recommendation on loading the template manually? Generate first with `index_template` binary and then load manually?
* Do we ship in index_template binary similar to import_dashboards? Can we integrate it into beats directly? -> setup
* Should template config be moved to the top level (like `dashboard`)
Part of elastic#3654
19a4e0d to
9d2a5c8
Compare
3eaa456 to
8c3cb2e
Compare
|
jenkins, retest it |
So far the template was loaded from the
beatname.template.jsonfiles. This now uses thefields.ymlfile to dynamically generate the template based on the ES version connected to. The name of the template is set dynamically based ontemplate.name. By default it has the value {beatname}.Part of #3654