Add basic telemetry to collector#284
Conversation
This change adds a Prometheus endpoint using OC to provide metrics to the collector. There are settings to control the level and port in which the metrics are exposed. The main metrics added for now are in the queued exporters.
| <-signalsChannel | ||
| logger.Info("Starting shutdown...") | ||
|
|
||
| // TODO: orderly shutdown: first receivers, then flushing pipelines giving |
There was a problem hiding this comment.
at this point, shouldn't you also gracefully shutdown the http server that is serving metrics?
There was a problem hiding this comment.
(shutdown would ideally include waiting the http server to exit before exiting the process)
There was a problem hiding this comment.
To gracefully shutdown here we will need to start having a timeout and so on, just calling Close is a rude shutdown (ie.: underlying listener is closed). This is desirable but we didn't do it yet for the pipes sending data to the backend. I think that adding it now for the handler of Prometheus metrics is an overkill (right now the worst that can happen is a GET to be cut in the middle and Prometheus losing one last scrape of data, but that was already in a race).
There was a problem hiding this comment.
Makes sense. In the long run when more sophisticated graceful shutdown is desired, you could consider returning a "closeFn" from initTelemetry that is appended to existing list.
* Add basic telemetry to collector This change adds a Prometheus endpoint using OC to provide metrics to the collector. There are settings to control the level and port in which the metrics are exposed. The main metrics added for now are in the queued exporters. * Check for view registration errors * PR Feedback
This change adds a Prometheus endpoint using OC to provide metrics
to the collector. There are settings to control the level and port
in which the metrics are exposed. The main metrics added for now
are in the queued exporters.