Skip to content

Prometheus data collector#3213

Merged
tsg merged 1 commit intoelastic:masterfrom
ruflin:prometheus-collector
Dec 21, 2016
Merged

Prometheus data collector#3213
tsg merged 1 commit intoelastic:masterfrom
ruflin:prometheus-collector

Conversation

@ruflin
Copy link
Copy Markdown
Contributor

@ruflin ruflin commented Dec 16, 2016

Generic data collector metricset for prometheus

Further improvements should be made to the data model, for example group same keys and handlers into one event.

@ruflin ruflin added in progress Pull request is currently in progress. Metricbeat Metricbeat labels Dec 16, 2016
@tsg
Copy link
Copy Markdown
Contributor

tsg commented Dec 16, 2016

This is a good start.

On the data model, I think it would be better to have the field name as the metric name (without the labels), so aggregations can be applied across metrics that differ only by labels. In other words, transform the labels into Metricbeat fields.

I know that's more complicated because it means we need to split them in different documents, but should be doable.

@otisg
Copy link
Copy Markdown

otisg commented Dec 18, 2016

@ruflin @tsg What's the motivation for "monitoring the monitor" (Prometheus being the monitoring tool itself)?

@tsg
Copy link
Copy Markdown
Contributor

tsg commented Dec 19, 2016

@otisg The Prometheus module contains two metricsets at the moment. One is the status which is indeed monitoring the Prometheus server (so monitoring the monitor). The other is the collector which can query any Prometheus exporter or any app that offers a Prometheus endpoint. In this case, you can use Metricbeat + Elasticsearch as an alternative to Prometheus.

@ruflin ruflin force-pushed the prometheus-collector branch from b15d005 to 52f089a Compare December 19, 2016 12:28
@ruflin ruflin added review and removed in progress Pull request is currently in progress. labels Dec 19, 2016
@ruflin ruflin force-pushed the prometheus-collector branch from 52f089a to 1de2f20 Compare December 19, 2016 12:33
Generic data collector metricset for prometheus. This metricset allows to fetch data from the prometheus exporters: https://prometheus.io/docs/instrumenting/exporters/

The fields.yml is empty as we don't know the namespace and the fields in advance.

Potential future improvements:

* Convert all _ to .
* Convert _sum and _count to .sum and .count with the same parent key
@ruflin ruflin force-pushed the prometheus-collector branch from 1de2f20 to 661b89f Compare December 19, 2016 16:21
description: >
Event round trip time in microseconds.

- name: metricset.namespace
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this entry meant to be an example? Does it make sense to include it? Seems to me it will cause more confusion.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, nevermind, it's the actual namespace.

@tsg tsg merged commit bf797b3 into elastic:master Dec 21, 2016
@ruflin ruflin deleted the prometheus-collector branch January 13, 2017 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants