Skip to content

[Monitoring] Use op_type = create with ES version >= 7.5.0#14313

Merged
ycombinator merged 4 commits intoelastic:masterfrom
ycombinator:lb-mon-bulk-optype-create
Nov 1, 2019
Merged

[Monitoring] Use op_type = create with ES version >= 7.5.0#14313
ycombinator merged 4 commits intoelastic:masterfrom
ycombinator:lb-mon-bulk-optype-create

Conversation

@ycombinator
Copy link
Copy Markdown
Contributor

@ycombinator ycombinator commented Oct 29, 2019 •
edited
Loading

Follow up to #13936.

In #13936, we taught the Elasticsearch output client in libbeat to use the create op_type in bulk requests to Elasticsearch if the Elasticsearch cluster's version was 7.5.0 or newer. This PR similarly teaches the Elasticsearch monitoring client to do the same.

Here's what the bulk request from the monitoring client looks like with Elasticsearch 7.4.0:

7_4_0

And here's what it looks like with Elasticsearch 8.0.0:

8_0_0

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Oct 29, 2019

Pinging @elastic/stack-monitoring (Stack monitoring)

@ycombinator ycombinator requested review from ph and urso October 29, 2019 19:22
@ycombinator ycombinator marked this pull request as ready for review October 29, 2019 19:25
@ycombinator ycombinator changed the title Use op_type = create with ES version >= 7.5.0 [Monitoring] Use op_type = create with ES version >= 7.5.0 Oct 29, 2019
@ycombinator ycombinator mentioned this pull request Oct 29, 2019
Merged
ph
ph approved these changes Oct 29, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@ph ph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread libbeat/monitoring/report/elasticsearch/client.go Outdated
Copy link
Copy Markdown
Contributor

@ph ph Oct 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't know about lessThan make it a lot cleaner.

@ycombinator
Copy link
Copy Markdown
Contributor Author

ycombinator commented Oct 29, 2019

CI failures look related to this PR. Will investigate and fix.

@ycombinator ycombinator added in progress Pull request is currently in progress. v7.5.1 and removed review v7.5.1 labels Oct 30, 2019
@ycombinator
Copy link
Copy Markdown
Contributor Author

ycombinator commented Oct 30, 2019

I was unable to reproduce the CI failures, specifically the test_monitoring.py system test in libbeat, locally. I compared the SHA256 digests of the elasticsearch:8.0.0-SNAPSHOT image that was downloaded to my local machine with the one downloaded by CI and they are the same: 3be50a3164826679d3a9aa7aab15e45de75d87f6be5d3876982b042eb45a3862. So I'm not sure why this is only failing on CI. 🤔

@ycombinator
Copy link
Copy Markdown
Contributor Author

ycombinator commented Oct 30, 2019

jenkins, test this

@ycombinator
Copy link
Copy Markdown
Contributor Author

ycombinator commented Nov 1, 2019

Now the error seems to have disappeared from CI too. I want to make sure this is not some kind of flakiness in the test, so I'm going to re-run CI a few times on this PR.

@ycombinator
Copy link
Copy Markdown
Contributor Author

ycombinator commented Nov 1, 2019

jenkins, test this

1 similar comment
@ycombinator
Copy link
Copy Markdown
Contributor Author

ycombinator commented Nov 1, 2019

jenkins, test this

@ycombinator
Copy link
Copy Markdown
Contributor Author

ycombinator commented Nov 1, 2019

Travis CI is green. Jenkins CI is red because of x-pack/agent builds being yellow, which is unrelated to this PR. Merging.

@ycombinator ycombinator merged commit abbe7fb into elastic:master Nov 1, 2019
@ycombinator ycombinator deleted the lb-mon-bulk-optype-create branch November 1, 2019 18:46
@ycombinator ycombinator mentioned this pull request Nov 1, 2019
Merged
@cwurm cwurm mentioned this pull request Nov 1, 2019
Merged
ycombinator added a commit that referenced this pull request Nov 7, 2019
@ycombinator
[Monitoring] Use op_type = create with ES version >= 7.5.0 (#14313) (#…
b42019f 
…14372)

* Use op_type = create with ES version >= 7.5.0

* Initialize map

* Printing out logs for debugging

* Removing debugging logs
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
@ycombinator
[Monitoring] Use op_type = create with ES version >= 7.5.0 (elastic#1…
87ad0f4 
…4313) (elastic#14372)

* Use op_type = create with ES version >= 7.5.0

* Initialize map

* Printing out logs for debugging

* Removing debugging logs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@ph ph ph approved these changes

@urso urso urso approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Feature:Stack Monitoring in progress Pull request is currently in progress. libbeat v7.5.1 v7.6.0 v8.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ycombinator @elasticmachine @ph @urso