Skip to content

insights: enable the anomaly detector#86673

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
matthewtodd:insights-enable-anomaly-detector
Aug 25, 2022
Merged

insights: enable the anomaly detector#86673
craig[bot] merged 1 commit intocockroachdb:masterfrom
matthewtodd:insights-enable-anomaly-detector

Conversation

@matthewtodd
Copy link
Copy Markdown

This change turns on the insights "anomaly detector" by default,
catching statement executions in the >p99 latency for their fingerprint
that are also far enough away from median latency and above a (default
50ms) threshold.

Release justification: Category 2: Bug fixes and low-risk updates to new
functionality.

Release note (ops change): The sql.insights.anomaly_detection.enabled
cluster setting now defaults to true, and the
sql.insights.anomaly_detection.latency_threshold cluster setting now
defaults to 50ms, down from 100ms to complement the fixed-threshold
detector's default of 100ms.

@matthewtodd matthewtodd requested a review from a team August 23, 2022 15:49
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

This change turns on the insights "anomaly detector" by default,
catching statement executions in the >p99 latency for their fingerprint
that are also far enough away from median latency and above a (default
50ms) threshold.

Release justification: Category 2: Bug fixes and low-risk updates to new
functionality.

Release note (ops change): The `sql.insights.anomaly_detection.enabled`
cluster setting now defaults to true, and the
`sql.insights.anomaly_detection.latency_threshold` cluster setting now
defaults to 50ms, down from 100ms to complement the fixed-threshold
detector's default of 100ms.
Copy link
Copy Markdown
Contributor

@maryliag maryliag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the user fits both the anomaly detector and the "regular" detector, you will see one row for each, or just one row?

Reviewed 4 of 4 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @matthewtodd)

Copy link
Copy Markdown
Contributor

@maryliag maryliag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*if the query fits

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @matthewtodd)

Copy link
Copy Markdown
Author

@matthewtodd matthewtodd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good q! Just the one row. (That's the point of the compositeDetector.)

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @matthewtodd)

Copy link
Copy Markdown
Contributor

@maryliag maryliag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! :lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @matthewtodd)

@matthewtodd
Copy link
Copy Markdown
Author

bors r+

@matthewtodd
Copy link
Copy Markdown
Author

I watched the anomaly detector's memory usage while running the ycsb-b workload. With 22 total statement fingerprints, memory reached 37KiB then settled down at just under 15 KiB.

Our default memory limit of 1MiB means we would track somewhere in the range of 600-1500 distinct fingerprints.

Screen Shot 2022-08-24 at 12 58 06 PM

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 24, 2022

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 25, 2022

Build succeeded:

@craig craig bot merged commit 2787ff1 into cockroachdb:master Aug 25, 2022
@matthewtodd matthewtodd deleted the insights-enable-anomaly-detector branch August 25, 2022 00:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants