Optimize label handling with predefined label keys in counter/gauge by p0rtale · Pull Request #508 · tarantool/metrics

p0rtale · 2025-06-25T13:24:40Z

This change adds support for predefined label keys in counter and gauge metric constructors, optimizing label processing.

Changes:

Add optional label_keys table to counter() and gauge() initialization
Replace dynamic label sorting with direct key generation when label_keys are provided, eliminating unnecessary processing overhead

Dynamic labels still supported when label_keys is omitted.

Closes TNTP-3453

p0rtale · 2025-06-25T13:45:58Z

variable labels: 85.6 s
fixed labels: 36.5 s

Bench script

local metrics = require('metrics')
local clock = require('clock')
local log = require('log')

local GAUGES_COUNT = 75
local KEYS_COUNT = 10000
local OPS_COUNT = 50000000

local label_keys = {
    "service", "version", "env", "region", "status", "deployment"
}

local label_values = {
    service = { "auth", "cart", "payment", "storage", "analytics", "notify", "gateway" },
    version = { "v1.0", "v1.1", "v1.2", "v2.0", "v2.1", "v2.2", "v3.0-beta" },
    env = { "prod", "stage", "dev", "test", "canary" },
    region = { "eu-west", "us-east", "us-west", "asia-south", "asia-east" },
    status = { "active", "inactive", "draining", "booting" },
    deployment = { "blue", "green", "red", "canary" }
}

local function generate_random_labels()
    local labels = {}
    for _, key in ipairs(label_keys) do
        local values = label_values[key]
        labels[key] = values[math.random(1, #values)]
    end
    return labels
end

local label_combinations = {}
for _ = 1, KEYS_COUNT do
    table.insert(label_combinations, generate_random_labels())
end

local var_gauges = {}
local fixed_gauges = {}
for i = 1, GAUGES_COUNT do
    var_gauges[i] = metrics.gauge('var_gauge_' .. i, 'Description for gauge ' .. i)
    fixed_gauges[i] = metrics.gauge('fixed_gauge_' .. i, 'Description for gauge ' .. i, {}, label_keys)
end

local function run_bench(gauges)
    local start_time = clock.monotonic()

    for _ = 1, OPS_COUNT do
        local gauge_idx = math.random(1, #gauges)
        local label_idx = math.random(1, #label_combinations)
        local value = math.random(1, 1000)

        gauges[gauge_idx]:set(value, label_combinations[label_idx])
    end

    return clock.monotonic() - start_time
end

local t1 = run_bench(var_gauges)
local t2 = run_bench(fixed_gauges)
local t3 = run_bench(var_gauges)
local t4 = run_bench(fixed_gauges)

log.info("variable labels: %.10f s", (t1 + t3) / 2)
log.info("fixed labels: %.10f s", (t2 + t4) / 2)

DifferentialOrange

What's the motivation? Is it the optimization or more assertive API for metrics users? If it's the optimization, I think the solution can be expanded to remove redundant table allocation (parts = {}) or even table.concat. As far as I remember, they also take a lot of work.

We shouldn't forget to

add a CHANGELOG entry,
update the API documentation (like here).

DifferentialOrange · 2025-07-01T08:17:37Z

metrics/collectors/shared.lua

    if type(label_pairs) ~= 'table' then
        return ""
    end
+
+    if label_keys ~= nil then


Shouldn't it be

Suggested change

if type(label_pairs) ~= 'table' then

return ""

end

if label_keys ~= nil then

if (label_keys == nil) and (type(label_pairs) ~= 'table') then

return ""

end

if label_keys ~= nil then

?

If label_keys is not nil and label_pairs is invalid, we won't be able to make the key

So you're fine with ignoring invalid label_pairs, including nil?

Oh, I see what you mean now

metrics/collectors/shared.lua

DifferentialOrange

I don't have any more question to this implementation. There is also a field for improvement. For example, one can create parts table and compute keys length only once per collector. Maybe table.concat can be removed at all in spite of some pre-built format string. Is there actually a performance increase? And how much is it?

oleg-jukovec

Please, squash commits into a single one well-described commit.

@TarantoolBot

Bump metric package submodule to 1.6.1. Commits from PR[1-8] add and remove metrics to Tarantool. 1. tarantool/metrics#508 2. tarantool/metrics#516 3. tarantool/metrics#519 4. tarantool/metrics#521 5. tarantool/metrics#524 6. tarantool/metrics#525 7. tarantool/metrics#527 8. tarantool/metrics#529 Closes #11950 @TarantoolBot document Title: Bump metrics module to 1.6.1 `'tnt_memory'`, `'tnt_memory_virt'`, `'schema_needs_upgrade'` are new metrics, introduced in metrics 1.6.0, and enabled by default. `'tnt_cartridge_config_checksum'` is new metric, introduced in metrics 1.5.0. New optional ``label_keys`` parameter for ``counter()`` and ``gauge()`` metrics are presented in metrics 1.4.0.

@TarantoolBot

Bump metric package submodule to 1.6.1. Commits from PR[1-7] add and remove metrics to Tarantool. 1. tarantool/metrics#508 2. tarantool/metrics#516 3. tarantool/metrics#519 4. tarantool/metrics#521 5. tarantool/metrics#524 6. tarantool/metrics#527 7. tarantool/metrics#529 Closes #11950 @TarantoolBot document Title: Bump metrics module to 1.6.1 `'tnt_memory'`, `'tnt_memory_virt'`, `'schema_needs_upgrade'` are new metrics, introduced in metrics 1.6.0, and enabled by default. `'tnt_cartridge_config_checksum'` is new metric, introduced in metrics 1.5.0. New optional ``label_keys`` parameter for ``counter()`` and ``gauge()`` metrics are presented in metrics 1.4.0.

@TarantoolBot

Bump metric package submodule to 1.6.1. Commits from PR[1-7] add new metrics to Tarantool. 1. tarantool/metrics#508 2. tarantool/metrics#516 3. tarantool/metrics#519 4. tarantool/metrics#521 5. tarantool/metrics#524 6. tarantool/metrics#527 7. tarantool/metrics#529 Closes #11950 @TarantoolBot document Title: Bump metrics module to 1.6.1 `'tnt_memory'`, `'tnt_memory_virt'`, `'schema_needs_upgrade'` are new metrics, introduced in metrics 1.6.0, and enabled by default. `'tnt_cartridge_config_checksum'` is new metric, introduced in metrics 1.5.0. New optional ``label_keys`` parameter for ``counter()`` and ``gauge()`` metrics are presented in metrics 1.4.0.

@TarantoolBot

Bump metric package submodule to 1.6.1. Commits from PR[1-7] add new metrics to Tarantool. 1. tarantool/metrics#508 2. tarantool/metrics#516 3. tarantool/metrics#519 4. tarantool/metrics#521 5. tarantool/metrics#524 6. tarantool/metrics#527 7. tarantool/metrics#529 Closes #11950 @TarantoolBot document Title: Bump metrics module to 1.6.1 `'tnt_memory'`, `'tnt_memory_virt'`, `'schema_needs_upgrade'` are new metrics, introduced in metrics 1.6.0, and enabled by default. `'tnt_cartridge_config_checksum'` is new metric, introduced in metrics 1.5.0. New optional ``label_keys`` parameter for ``counter()`` and ``gauge()`` metrics are presented in metrics 1.4.0.

@TarantoolBot

Bump metric package submodule to 1.6.1. Commits from PR[1-7] add new metrics to Tarantool. 1. tarantool/metrics#508 2. tarantool/metrics#516 3. tarantool/metrics#519 4. tarantool/metrics#521 5. tarantool/metrics#524 6. tarantool/metrics#527 7. tarantool/metrics#529 Closes #11950 @TarantoolBot document Title: Bump metrics module to 1.6.1 `'tnt_memory'`, `'tnt_memory_virt'`, `'schema_needs_upgrade'` are new metrics, introduced in metrics 1.6.0, and enabled by default. `'tnt_cartridge_config_checksum'` is new metric, introduced in metrics 1.5.0. New optional ``label_keys`` parameter for ``counter()`` and ``gauge()`` metrics are presented in metrics 1.4.0.

@TarantoolBot

Bump metric package submodule to 1.6.1. Commits from PR[1-7] add new metrics to Tarantool. 1. tarantool/metrics#508 2. tarantool/metrics#516 3. tarantool/metrics#519 4. tarantool/metrics#521 5. tarantool/metrics#524 6. tarantool/metrics#527 7. tarantool/metrics#529 Closes #11950 @TarantoolBot document Title: Bump metrics module to 1.6.1 `'tnt_memory'`, `'tnt_memory_virt'`, `'schema_needs_upgrade'` are new metrics, introduced in metrics 1.6.0, and enabled by default. `'tnt_cartridge_config_checksum'` is new metric, introduced in metrics 1.5.0. New optional ``label_keys`` parameter for ``counter()`` and ``gauge()`` metrics are presented in metrics 1.4.0.

@TarantoolBot

Bump metric package submodule to 1.6.1. Commits from PR[1-7] add new metrics to Tarantool. 1. tarantool/metrics#508 2. tarantool/metrics#516 3. tarantool/metrics#519 4. tarantool/metrics#521 5. tarantool/metrics#524 6. tarantool/metrics#527 7. tarantool/metrics#529 Closes #11950 @TarantoolBot document Title: Bump metrics module to 1.6.1 `'tnt_memory'`, `'tnt_memory_virt'`, `'schema_needs_upgrade'` are new metrics, introduced in metrics 1.6.0, and enabled by default. `'tnt_cartridge_config_checksum'` is new metric, introduced in metrics 1.5.0. New optional ``label_keys`` parameter for ``counter()`` and ``gauge()`` metrics are presented in metrics 1.4.0.

@TarantoolBot

Bump metric package submodule to 1.6.1. Commits from PR[1-7] add new metrics to Tarantool. 1. tarantool/metrics#508 2. tarantool/metrics#516 3. tarantool/metrics#519 4. tarantool/metrics#521 5. tarantool/metrics#524 6. tarantool/metrics#527 7. tarantool/metrics#529 Closes #11950 @TarantoolBot document Title: Bump metrics module to 1.6.1 `'tnt_memory'`, `'tnt_memory_virt'`, `'schema_needs_upgrade'` are new metrics, introduced in metrics 1.6.0, and enabled by default. `'tnt_cartridge_config_checksum'` is new metric, introduced in metrics 1.5.0. New optional ``label_keys`` parameter for ``counter()`` and ``gauge()`` metrics are presented in metrics 1.4.0.

@TarantoolBot

Bump metric package submodule to 1.6.1. Commits from PR[1-7] add new metrics to Tarantool. 1. tarantool/metrics#508 2. tarantool/metrics#516 3. tarantool/metrics#519 4. tarantool/metrics#521 5. tarantool/metrics#524 6. tarantool/metrics#527 7. tarantool/metrics#529 Closes #11950 @TarantoolBot document Title: Bump metrics module to 1.6.1 `'tnt_memory'`, `'tnt_memory_virt'`, `'schema_needs_upgrade'` are new metrics, introduced in metrics 1.6.0, and enabled by default. `'tnt_cartridge_config_checksum'` is new metric, introduced in metrics 1.5.0. New optional ``label_keys`` parameter for ``counter()`` and ``gauge()`` metrics are presented in metrics 1.4.0.

@TarantoolBot

Bump metric package submodule to 1.6.1. Commits from PR[1-7] add new metrics to Tarantool. 1. tarantool/metrics#508 2. tarantool/metrics#516 3. tarantool/metrics#519 4. tarantool/metrics#521 5. tarantool/metrics#524 6. tarantool/metrics#527 7. tarantool/metrics#529 Closes #11950 @TarantoolBot document Title: Bump metrics module to 1.6.1 `'tnt_memory'`, `'tnt_memory_virt'`, `'schema_needs_upgrade'` are new metrics, introduced in metrics 1.6.0, and enabled by default. `'tnt_cartridge_config_checksum'` is new metric, introduced in metrics 1.5.0. New optional ``label_keys`` parameter for ``counter()`` and ``gauge()`` metrics are presented in metrics 1.4.0.

a1div0 assigned p0rtale Jun 27, 2025

a1div0 approved these changes Jun 27, 2025

View reviewed changes

a1div0 requested a review from DifferentialOrange June 27, 2025 07:30

DifferentialOrange reviewed Jun 27, 2025

View reviewed changes

p0rtale requested a review from DifferentialOrange June 29, 2025 20:27

DifferentialOrange reviewed Jul 1, 2025

View reviewed changes

p0rtale requested a review from DifferentialOrange July 1, 2025 09:58

p0rtale force-pushed the fast-gauge branch from ded251c to fcf0f47 Compare July 7, 2025 10:09

DifferentialOrange reviewed Jul 7, 2025

View reviewed changes

metrics/collectors/shared.lua Outdated Show resolved Hide resolved

DifferentialOrange approved these changes Jul 7, 2025

View reviewed changes

oleg-jukovec self-requested a review July 7, 2025 10:42

oleg-jukovec approved these changes Jul 7, 2025

View reviewed changes

oleg-jukovec requested changes Jul 7, 2025

View reviewed changes

Optimize label handling with predefined label keys in counter/gauge

e15b565

p0rtale force-pushed the fast-gauge branch from aad176e to e15b565 Compare July 7, 2025 11:11

p0rtale requested a review from oleg-jukovec July 7, 2025 11:11

oleg-jukovec approved these changes Jul 7, 2025

View reviewed changes

oleg-jukovec merged commit 1f81509 into tarantool:master Jul 7, 2025

patapenka-alexey mentioned this pull request Oct 22, 2025

lua: bump metrics module tarantool/tarantool#11950

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optimize label handling with predefined label keys in counter/gauge#508

Optimize label handling with predefined label keys in counter/gauge#508
oleg-jukovec merged 1 commit intotarantool:masterfrom
p0rtale:fast-gauge

p0rtale commented Jun 25, 2025

Uh oh!

p0rtale commented Jun 25, 2025 •

edited

Loading

Uh oh!

DifferentialOrange left a comment •

edited

Loading

Uh oh!

DifferentialOrange Jul 1, 2025

Uh oh!

p0rtale Jul 1, 2025

Uh oh!

DifferentialOrange Jul 1, 2025

Uh oh!

p0rtale Jul 1, 2025

Uh oh!

Uh oh!

Uh oh!

DifferentialOrange left a comment •

edited

Loading

Uh oh!

oleg-jukovec left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

p0rtale commented Jun 25, 2025

Uh oh!

p0rtale commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DifferentialOrange left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DifferentialOrange Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

p0rtale Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

DifferentialOrange Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

p0rtale Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

DifferentialOrange left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oleg-jukovec left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

p0rtale commented Jun 25, 2025 •

edited

Loading

DifferentialOrange left a comment •

edited

Loading

DifferentialOrange left a comment •

edited

Loading