Skip to content

ui: heatmap visualization for histogram buckets#13096

Merged
beorn7 merged 4 commits intoprometheus:mainfrom
Loori-R:support-heatmap
Nov 24, 2023
Merged

ui: heatmap visualization for histogram buckets#13096
beorn7 merged 4 commits intoprometheus:mainfrom
Loori-R:support-heatmap

Conversation

@Loori-R
Copy link
Copy Markdown
Contributor

@Loori-R Loori-R commented Nov 3, 2023

Description:

This Pull Request implements a heatmap visualization feature for histogram buckets on the Prometheus /graph page. It utilizes a flotjs plugin for rendering, aligning its functionality with that observed in the VictoriaMetrics UI (vmui), as discussed in issue: VictoriaMetrics#3384.

Key Changes:

  • flotjs Plugin Integration: Facilitates the rendering of heatmaps.
  • "Show Histogram Graph" Button Added: A new button placed adjacent to the "Show Stacked Graph" button.
  • Histogram Rendering: Clicking the "Show Histogram Graph" button renders the graph as a histogram.
  • Auto-switch to Line Graph: The graph automatically transitions to a linear representation if the query data lacks bucket data.
  • Button Activation Logic: The "Show Histogram Graph" button is enabled only when the data comprises a single histogram series.
  • Deactivation Logic: The button is disabled if the data includes more or fewer than one histogram series.

Future Enhancements:

  • Renderer Selection Option: Consider substituting the button with a selector to choose the histogram rendering type when multiple series are available.
Screenshot 2023-10-30 at 11 36 22

Ref #12995

Signed-off-by: Yury Moladau <yurymolodov@gmail.com>
@beorn7
Copy link
Copy Markdown
Member

beorn7 commented Nov 9, 2023

Thank you very much. Still on my review list. I'll have a look next week.

@juliusv your continued input would be appreciated in the meantime.

Copy link
Copy Markdown
Member

@beorn7 beorn7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functionality looks good to me. Thank you very much.

@juliusv up to you to review the actual js and tsx code. :)

* Inspired by a similar feature in VictoriaMetrics.
* See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3384 for more details.
* Developed by VictoriaMetrics team.
*/
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this should also state the license under which this code is included here? Or a LICENSE file in the same directory?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the MIT license to jquery.flot.heatmap.js

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I'm a bit confused because VictoriaMetrics is Apache 2 licensed and not MIT.

I first thought that maybe the part of the codebase where this file is copied from is MIT licensed. But I couldn't even find a file that is the same (or at least similar) to this one.

Where did you copy this file from?

Or alternatively, if you created this file just for this PR, then we don't really need to put it in a vendor directory (as it isn't vendored), and we can use the usual Apache 2 license and don't need a copyright header.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was created for this PR. I saw that all plugins for flot were in this folder, so I placed it there. I can create a new folder, for example flot/plugins, and move it there, or you can suggest a better place for it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see… On the other hand, the other files are actually directly copied from another repository.

Not sure what to do. I'm not very familiar with the UI code. @juliusv could you advise how to handle the case where a flot plugin has been created originally and not just copied from another source? Should we still have it here or in another directory?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nexucis might also be able to help here.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind this particular part too much, IMO we can keep it in this location for now.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I agree we can keep it here. For the moment I'm just sad to see another file going there since my best wish so far is to drop this vendor folder. But that's definitely another subject totally out of scope of this PR

Signed-off-by: Yury Moladau <yurymolodov@gmail.com>
Copy link
Copy Markdown
Member

@juliusv juliusv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall nice, thanks! Just a couple of comments, mostly around readability/simplicity, which are hopefully easy to resolve.

import { GraphExemplar, GraphProps, GraphSeries } from './Graph';

export function isHistogramData(data: GraphProps['data']) {
if (!data?.result?.length) return false;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and elsewhere: We generally use a style where we always use curly braces and separate lines for blocks, even if they are just one line like here. That way it's easier to see where control flow happens and easier / less error-prone to extend when necessary.

if (!data?.result?.length) return false;
const result = data.result;
if (result.length < 2) return false;
const histogramLabels = ['le'];
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't need an array here - that was only necessary in the VictoriaMetrics case, where they have multiple different histogram labels. I'd just go with const bucketLabel = 'le';.

Would make the code easier to read too, for example:

result.every((r) => histogramLabels.some((l) => l in r.metric));

would become

result.every((r) => bucketLabel in r.metric));

or since you do it in other places already, feel free to hardcode the le instead of having a constant for it:

result.every((r) => r.metric.le));

IMO that's even clearer to read :)

if (!buckets.every((a) => a.labels.le)) return buckets;

const sortedBuckets = buckets.sort((a, b) => promValueToNumber(a.labels.le) - promValueToNumber(b.labels.le));
let prevBucket: GraphSeries | GraphExemplar = { data: [[0, 0]], labels: {}, color: '', index: 0 };
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can just be:

Suggested change
let prevBucket: GraphSeries | GraphExemplar = { data: [[0, 0]], labels: {}, color: '', index: 0 };
let prevBucket: GraphSeries = { data: [[0, 0]], labels: {}, color: '', index: 0 };

const result: GraphSeries[] = [];
let index = 0;

for (const bucket of sortedBuckets) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you have index = 0 and index += 1, how about using a classic for loop while at the same time eliminating the need for a prevBucket variable, like this?

  for (let i = 0; i < sortedBuckets.length; i++) {
    const values = [];
    const { data, labels, color } = sortedBuckets[i];

    for (const [timestamp, value] of data) {
      const prevVal = sortedBuckets[i - 1]?.data.find((v) => v[0] === timestamp)?.[1] || 0;
      const newVal = Number(value) - +prevVal;
      values.push([Number(timestamp), newVal]);
    }

    result.push({
      data: values,
      labels,
      color,
      index: i,
    });
  }

endTime: number | null; // Timestamp in milliseconds.
resolution: number | null; // Resolution in seconds.
stacked: boolean;
histogram: boolean;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer calling this heatmap or histogramHeatmap for clarity in all the dependent code that I read. Just histogram makes me think that it just means that the underlying data is a histogram, or if I think about it as a display option, that it means a regular histogram bar chart.

// TODO: Currently, this method toggles histogram rendering on and off.
// Consider extending this to allow selection of a specific series from
// multiple available ones for histogram rendering in the future.
this.setOptions({ histogram: !this.props.options.histogram, stacked: false });
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This clarifies to me that stacked is not remembered across switching into heatmap mode and back - I think that's ok, but then it would (at least eventually) make more sense to replace the stacked boolean option with a displayMode: 'lines' | 'stacked' | 'heatmap' option. Then we don't have multiple option fields where some values might be invalid state combinations.

</Button>
{/* TODO: Consider replacing this button with a select dropdown in the future,
to allow users to choose from multiple histogram series if available. */}
<Button
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of enabling/disabling this button, I'd prefer only showing it at all when the data is a histogram. That way users who never deal with histograms don't even have to think about it, while users who do deal with histograms will notice it even more when it only appears for histogram data.

Signed-off-by: Yury Moladau <yurymolodov@gmail.com>
@Loori-R
Copy link
Copy Markdown
Contributor Author

Loori-R commented Nov 24, 2023

@juliusv I've fixed the stuff you mentioned in the comments, in commit 50c3c5a

Copy link
Copy Markdown
Member

@juliusv juliusv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thank you! Just one last comment around the URL options.


case 'stacked':
return { stacked: decodedValue === '1' };
return { displayMode: decodedValue === '1' ? GraphDisplayMode.Stacked : GraphDisplayMode.Lines };
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will always want to encode all available graph options in the URL so that we can send around heatmap graphs as well, so we'll want to encode the new display mode setting as well.

I'd go for this approach:

  • On reading, be backwards-compatible with the legacy stacked parameter and just translate it to the new display mode setting as you are doing it here already.
  • On writing the options back to the URL, only store the display mode parameter (maybe display_mode=lines|stacked|heatmap?) and omit stacked.

If a URL happens to somehow contain both the legacy stacked parameter and display_mode, then display_mode can take precedence.

That way a new Prometheus version can still open the old URLs correctly, but will produce the new format going forward.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think, what's the better practice for test cases: using string literals or enums for parameters like 'display_mode'?

.toEqual(...g0.display_mode=${GraphDisplayMode.Stacked}&...);
or
.toEqual(...g0.display_mode=stacked&...);"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I get the question for tests, but I don't think it makes much of a difference in this kind of case, so I'm happy with either.

Signed-off-by: Yury Moladau <yurymolodov@gmail.com>
@juliusv
Copy link
Copy Markdown
Member

juliusv commented Nov 24, 2023

👍 Nice, thank you! Will merge once the tests are green :)

@beorn7 beorn7 merged commit 2e205ee into prometheus:main Nov 24, 2023
@beorn7
Copy link
Copy Markdown
Member

beorn7 commented Nov 24, 2023

I did that for you. 😄

Thank you very much @Loori-R .

@juliusv
Copy link
Copy Markdown
Member

juliusv commented Nov 24, 2023

@beorn7 Haha yes, thank you <3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants