Skip to content

fix(tsdb): appender does not honor append order#16823

Closed
krajorama wants to merge 2 commits intomainfrom
krajo/fix-commit-order
Closed

fix(tsdb): appender does not honor append order#16823
krajorama wants to merge 2 commits intomainfrom
krajo/fix-commit-order

Conversation

@krajorama
Copy link
Member

@krajorama krajorama commented Jul 2, 2025

Fixes: #14172
Fixes: #15177

Instead of 3 separate lists, keep a single list of samples with series reference. To support different data types I did a union type where floats are handled separately from complex types, but complex types are kept in one place.

Not optimized, and doesn't fix that WAL/WBL that keeps 5 different value types separately: floats, integer/float native histograms with standard/custom buckets, meaning that on reload the problem comes back.

Ideas for the WAL/WBL:

  1. introduce a new union record type that can efficiently encode all kinds of data (floats, integer/float native histograms with standard/custom buckets) and has room for mote types like native summaries.
  2. Keep existing record types, but ensure the order on readback by cleverly splitting samples between records - relying on the fact that mixing types is rare.

Fixes: #15177

Instead of 3 separate lists, keep a single list of samples with series
reference. To support different data types I did a union type where
floats are handled separately from complex types, but complex types are
kept in one place.

Not finished, in particular the WAL format is kept and there's a
conversion from the union type to separate record types for the WAL/WBL.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
@krajorama krajorama force-pushed the krajo/fix-commit-order branch from 17b70cf to 7876c57 Compare July 3, 2025 07:08
Add a testcase to the PromQL unit tests that has mixed order of floats
vs native histograms.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
@krajorama
Copy link
Member Author

superseded by #17071

@krajorama krajorama closed this Sep 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TSDB Commit should honor ordering between floats and native histograms promql engine does not return expected results with mixed floats+histograms

1 participant