Make logstash/node_stats metricset work for Stack Monitoring without xpack.enabled flag by sayden · Pull Request #21546 · elastic/beats

sayden · 2020-10-05T21:44:50Z

Ready to test

elasticmachine · 2020-10-05T21:44:52Z

Pinging @elastic/stack-monitoring (Stack monitoring)

elasticmachine · 2020-10-05T21:44:52Z

Pinging @elastic/integrations-services (Team:Services)

elasticmachine · 2020-10-05T22:02:16Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #21546 updated
Start Time: 2020-12-15T16:35:40.600+0000
Duration: 66 min 38 sec

Test stats 🧪

Test	Results
Failed	0
Passed	2277
Skipped	524
Total	2801

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	2277
Skipped	524
Total	2801

chrisronline · 2020-10-23T18:15:05Z

I'm seeing this error when running this:

2020-10-23T14:14:48.531-0400 ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(https://localhost:9200)): Connection marked as failed because the onConnect callback failed: error loading template: could not load template. Elasticsearch returned: couldn't load template: 400 Bad Request: {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"Invalid [path] value [logstash.node.stats.events.duration_in_millis] for field alias [logstash.stats.events.duration.ms]: an alias must refer to an existing field in the mappings."}],"type":"mapper_parsing_exception","reason":"Invalid [path] value [logstash.node.stats.events.duration_in_millis] for field alias [logstash.stats.events.duration.ms]: an alias must refer to an existing field in the mappings."},"status":400}. Response body: {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"Invalid [path] value [logstash.node.stats.events.duration_in_millis] for field alias [logstash.stats.events.duration.ms]: an alias must refer to an existing field in the mappings."}],"type":"mapper_parsing_exception","reason":"Invalid [path] value [logstash.node.stats.events.duration_in_millis] for field alias [logstash.stats.events.duration.ms]: an alias must refer to an existing field in the mappings."},"status":400}.

chrisronline

LGTM! UI loads up as expected!

…sh/node_stats_xpack_flag

…sh/node_stats_xpack_flag # Conflicts: # metricbeat/module/logstash/node_stats/data_xpack.go

ycombinator · 2020-12-15T11:09:26Z

metricbeat/module/logstash/node/node.go

-
-func (m *MetricSet) init() error {
-	if m.XPack {
-		return m.CheckPipelineGraphAPIsAvailable()


I think we will need to call m.CheckPipelineGraphAPIsAvailable() every time now, right, since we plan to always call the Logstash Pipeline Graph APIs in the Fetch() method, no?

I removed it because I didn't see any call to this method anywhere, haha. Anyways I guess it should be included so I added it back into the Fetch call directly.

ycombinator

This is looking mostly good. Just left one comment/question.

ycombinator

I left a comment/question a few minutes ago. Besides that, everything in this PR LGTM. So I'm approving it for merge now. Feel free to address my comment before merging or after, in the feature branch, as you prefer.

…I is available

This reverts commit da828ea.

…xpack.enabled flag (elastic#21546)

Atomic commit

eb3032c

sayden added Metricbeat Metricbeat Feature:Stack Monitoring Team:Services (Deprecated) Label for the former Integrations-Services team labels Oct 5, 2020

sayden self-assigned this Oct 5, 2020

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Oct 5, 2020

Upgrade mapping and data.json

20392da

sayden mentioned this pull request Nov 3, 2020

Make logstash/node metricset work for Stack Monitoring without xpack.enabled flag #21545

Closed

Fix test that got broken

63cd98c

chrisronline approved these changes Nov 6, 2020

View reviewed changes

sayden added 5 commits December 4, 2020 12:25

Merge branch 'feature-stack-monitoring-mb-ecs' into feature/mb/logsta…

f499698

…sh/node_stats_xpack_flag

Fix some tests and add a TestData test

d846639

Merge branch 'feature-stack-monitoring-mb-ecs' into feature/mb/logsta…

36c3a3e

…sh/node_stats_xpack_flag # Conflicts: # metricbeat/module/logstash/node_stats/data_xpack.go

Merge changes from master

9edd2c8

Cleaning things up

d7e5860

sayden force-pushed the feature/mb/logstash/node_stats_xpack_flag branch from 4ebe049 to d7e5860 Compare December 9, 2020 16:21

sayden added 6 commits December 10, 2020 20:36

Fix broken test

a2a23d2

Fix broken test

4b39731

Skip Python test to check documented fields

3a02dd9

Revert a hack to run tests without compose

dacf115

Fix duplicated field mapping

1044d9f

Fix indentation issue

de24dd0

sayden requested a review from ycombinator December 15, 2020 10:17

Fix duplicated field

4066b89

ycombinator reviewed Dec 15, 2020

View reviewed changes

ycombinator approved these changes Dec 15, 2020

View reviewed changes

sayden added 4 commits December 15, 2020 16:15

Add a check at the beginning of the Fetch function to see if Graph AP…

85ca839

…I is available

Update metricset too

da828ea

Revert "Update metricset too"

53db83e

This reverts commit da828ea.

Fix linting

f73b1ea

sayden merged commit 11f8738 into elastic:feature-stack-monitoring-mb-ecs Dec 15, 2020

sayden mentioned this pull request Apr 27, 2021

[Metricbeat] Remove xpack enabled flag on ES, Logstash, Beats and Kibana #24427

Merged

6 tasks

leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023

Make logstash/node_stats metricset work for Stack Monitoring without …

3dce95a

…xpack.enabled flag (elastic#21546)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make logstash/node_stats metricset work for Stack Monitoring without xpack.enabled flag#21546

Make logstash/node_stats metricset work for Stack Monitoring without xpack.enabled flag#21546
sayden merged 19 commits intoelastic:feature-stack-monitoring-mb-ecsfrom
sayden:feature/mb/logstash/node_stats_xpack_flag

sayden commented Oct 5, 2020

Uh oh!

elasticmachine commented Oct 5, 2020

Uh oh!

elasticmachine commented Oct 5, 2020

Uh oh!

elasticmachine commented Oct 5, 2020 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

Test stats 🧪

Uh oh!

chrisronline commented Oct 23, 2020

Uh oh!

chrisronline left a comment

Uh oh!

ycombinator Dec 15, 2020

Uh oh!

sayden Dec 15, 2020

Uh oh!

ycombinator left a comment

Uh oh!

ycombinator left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

sayden commented Oct 5, 2020

Uh oh!

elasticmachine commented Oct 5, 2020

Uh oh!

elasticmachine commented Oct 5, 2020

Uh oh!

elasticmachine commented Oct 5, 2020 • edited by jenkins-beats-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

Test stats 🧪

Uh oh!

chrisronline commented Oct 23, 2020

Uh oh!

chrisronline left a comment

Choose a reason for hiding this comment

Uh oh!

ycombinator Dec 15, 2020

Choose a reason for hiding this comment

Uh oh!

sayden Dec 15, 2020

Choose a reason for hiding this comment

Uh oh!

ycombinator left a comment

Choose a reason for hiding this comment

Uh oh!

ycombinator left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

elasticmachine commented Oct 5, 2020 •

edited by jenkins-beats-ci bot

Loading