rpc: bump threshold for latency jump reporting#98580
rpc: bump threshold for latency jump reporting#98580craig[bot] merged 1 commit intocockroachdb:masterfrom
Conversation
|
It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR? 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
|
This change is |
For months I've seen this misfire in nearly every single log line I've looked at, and I've had to grep it out in many L2 incidents. Maybe it works better when we suppress it for latencies <=50ms. Touches cockroachdb#96262. Fixes cockroachdb#98066. Epic: none Release note: None
erikgrinaker
left a comment
There was a problem hiding this comment.
Considering this measures heartbeat latencies, and they're very vulnerable to head-of-line blocking on the DefaultClass (thus not an accurate representation of the actual network latency), a high threshold of 50 ms seems appropriate.
Should we consider only measuring this on SystemClass while we're at it, to get better signal?
Not a bad idea but it also isn't a trivial change, so I'll pass. bors r=erikgrinaker |
|
Build failed: |
|
TestLossOfQuorumRecover cc @aliher1911 bors r=erikgrinaker |
|
Build succeeded: |
For months I've seen this misfire in nearly every single log line I've
looked at, and I've had to grep it out in many L2 incidents.
Maybe it works better when we suppress it for latencies <=50ms.
Touches #96262.
Fixes #98066.
Epic: none
Release note: None