Skip to content

Per tenant head_series_limit can't be set to unlimited (0) #6393

@lasermoth

Description

@lasermoth

When setting a head_series_limit to unlimited (0) for a given tenant, it fails to treat this as unlimited.

The following example is currently not possible:

write:
  global:
    max_concurrency: 30
    meta_monitoring_url: "http://localhost:9090"
    meta_monitoring_limit_query: "sum(prometheus_tsdb_head_series) by (tenant)"
  default:
    request:
      size_bytes_limit: 1024
      series_limit: 1000
      samples_limit: 10
    head_series_limit: 1000
  tenants:
    acme:
      request:
        size_bytes_limit: 0
        series_limit: 0
        samples_limit: 0
      head_series_limit: 0

It will result in for example:

"receive-limiter","currentSeries":46496,"level":"error","limit":0,"msg":"tenant above limit",

Looking at the function here

It only checks if the limitsPerTenant length is 0 and if the default limit is 0 (unlimited).

if len(h.limitsPerTenant) == 0 && h.defaultLimit == 0 {
 return true, nil
}

However if the tenant config is not empty, len > 0, we continue to evaluate the value agains the current value from the metamonitoring, which if 0 will always fail.

It should also check if the configured limit for the tenant is 0.

// If tenant limit is 0 we treat it as unlimited.
if limit == 0 {
  return true, nil
}

This helps where you would want to have lower set default limits to prevent abuse, but allow a specific tenant to have an unlimited value.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions