Skip to content

Kubernetes Apiserver metricset collects deprecated metrics #31834

@MichaelKatsoulis

Description

@MichaelKatsoulis

Kubernetes API server metricset collects metrics that are deprecated(apiserver_request_latencies, apiserver_request_count) in newer versions of kubernetes.
At the same time we don't collect some really important metrics(apiserver_request_duration_seconds_sum, apiserver_request_duration_seconds_count).

Currently we support k8s versions starting from 1.21.x so there is no need to keep searching for older versions' fields in current/newer versions of beats.

  1. All API server exposed metrics should be investigated and the one we collect should be reconsider.

  2. The API server dashboard is half broken because it tries to make use of the deprecated fields. It should be updated as well with more meaningful panels.

  3. Some metrics exposed from API server prometheus endpoint are of histogram type.
    apiserver_request_duration_seconds is one of them. We should find a way to store them as object_type: histogram in Elasticsearch so that we can leverage percentiles aggregation in Kibana for meaningful dashboard creation. This has been achieved in the past in lightweight modules like istio(link). It makes use of the use_types config option which allows prometheus collector metricset to map prometheus histogram types to elasticsearch histograms (PR). We need to find a way to leverage that feature(or the logic behind it) at a non-lightweight module for specific fields. (This bullet is moved to new issue Use Elasticsearch histogram type to store Prometheus histograms #32006)

Kubernetes Controller Manager metricset appears to have same issues.

Metadata

Metadata

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions