Skip to content

Add "synthetics-*-*" templates for synthetics fleet data#62193

Merged
dakrone merged 6 commits intoelastic:masterfrom
dakrone:add-synthetics-templates
Sep 14, 2020
Merged

Add "synthetics-*-*" templates for synthetics fleet data#62193
dakrone merged 6 commits intoelastic:masterfrom
dakrone:add-synthetics-templates

Conversation

@dakrone
Copy link
Copy Markdown
Member

@dakrone dakrone commented Sep 9, 2020

For the Elastic Agent we currently have logs and metrics, however, synthetic data doesn't belong
with those and thus we should have a place for it to live. This would be data reported from
heartbeat and under the 'monitoring' category.

This commit adds a composable index template for synthetics-*-* indices similar to the work in
#56709 and #57629.

Resolves #61665

For the Elastic Agent we currently have `logs` and `metrics`, however, synthetic data doesn't belong
with those and thus we should have a place for it to live. This would be data reported from
heartbeat and under the 'monitoring' category.

This commit adds a composable index template for `synthetics-*-*` indices similar to the work in
 elastic#56709 and elastic#57629.

Resolves elastic#61665
@dakrone dakrone added :Data Management/Indices APIs DO NOT USE. Use ":Distributed/Indices APIs" or ":StorageEngine/Templates" instead. v8.0.0 v7.10.0 labels Sep 9, 2020
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-core-features (:Core/Features/Indices APIs)

@elasticmachine elasticmachine added the Team:Data Management (obsolete) DO NOT USE. This team no longer exists. label Sep 9, 2020
Copy link
Copy Markdown
Contributor

@andreidan andreidan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Lee (it's only the docs tests that need some adjustments)

Copy link
Copy Markdown
Contributor

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM but I want some others to chime in too.

"hot": {
"actions": {
"rollover": {
"max_size": "50gb",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrewvc I assume this also works for synthetics for now?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH we haven't done enough benchmarking to know, but it seems reasonable

"priority": 100,
"data_stream": {},
"composed_of": [
"synthetics-mappings",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpountz I remember you wanted to "not" use components here. Perhaps it is a good time to adjust?

@ruflin
Copy link
Copy Markdown
Contributor

ruflin commented Sep 10, 2020

@ph @neptunian As soon as this lands, we also probably need to add a fake doc for the index pattern in Kibana.

@neptunian
Copy link
Copy Markdown

neptunian commented Sep 14, 2020

@ruflin if we expect to create an index pattern, like synthetics-* when a package installs we need to add this to our list of index pattern types. Then a placeholder would be created automatically.

@dakrone
Copy link
Copy Markdown
Member Author

dakrone commented Sep 14, 2020

@elasticmachine update branch

@dakrone
Copy link
Copy Markdown
Member Author

dakrone commented Sep 14, 2020

@elasticmachine run elasticsearch-ci/2

@dakrone dakrone merged commit d21ddb5 into elastic:master Sep 14, 2020
@dakrone dakrone deleted the add-synthetics-templates branch September 14, 2020 22:37
dakrone added a commit to dakrone/elasticsearch that referenced this pull request Sep 14, 2020
* Add "synthetics-*-*" templates for synthetics fleet data

For the Elastic Agent we currently have `logs` and `metrics`, however, synthetic data doesn't belong
with those and thus we should have a place for it to live. This would be data reported from
heartbeat and under the 'monitoring' category.

This commit adds a composable index template for `synthetics-*-*` indices similar to the work in
 elastic#56709 and elastic#57629.

Resolves elastic#61665
dakrone added a commit that referenced this pull request Sep 14, 2020
… (#62346)

* Add "synthetics-*-*" templates for synthetics fleet data

For the Elastic Agent we currently have `logs` and `metrics`, however, synthetic data doesn't belong
with those and thus we should have a place for it to live. This would be data reported from
heartbeat and under the 'monitoring' category.

This commit adds a composable index template for `synthetics-*-*` indices similar to the work in
 #56709 and #57629.

Resolves #61665
@ruflin
Copy link
Copy Markdown
Contributor

ruflin commented Sep 15, 2020

@neptunian SGTM

dakrone added a commit to dakrone/elasticsearch that referenced this pull request Sep 15, 2020
We need to ensure these are mapped as 'ip' instead of a keyword, even if they do end up not being
used.

Relates to elastic#62193
dakrone added a commit that referenced this pull request Sep 17, 2020
…2412)

We need to ensure these are mapped as 'ip' instead of a keyword, even if they do end up not being
used.

Relates to #62193
dakrone added a commit to dakrone/elasticsearch that referenced this pull request Sep 17, 2020
…astic#62412)

We need to ensure these are mapped as 'ip' instead of a keyword, even if they do end up not being
used.

Relates to elastic#62193
dakrone added a commit that referenced this pull request Sep 17, 2020
…gs (#62412) (#62553)

We need to ensure these are mapped as 'ip' instead of a keyword, even if they do end up not being
used.

Relates to #62193
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Data Management/Indices APIs DO NOT USE. Use ":Distributed/Indices APIs" or ":StorageEngine/Templates" instead. Team:Data Management (obsolete) DO NOT USE. This team no longer exists. v7.10.0 v8.0.0-alpha1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add new index for synthetics

8 participants