Skip to content

kvcoord: observability for in-flight writes and locking reads over intent budget#123340

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
miraradeva:mira-121471-obs
May 1, 2024
Merged

kvcoord: observability for in-flight writes and locking reads over intent budget#123340
craig[bot] merged 1 commit intocockroachdb:masterfrom
miraradeva:mira-121471-obs

Conversation

@miraradeva
Copy link
Copy Markdown
Contributor

@miraradeva miraradeva commented May 1, 2024

Since #121088, in-flight writes can include locking reads; because we don't estimate the size of the locks accurately for ranged locking reads, it is possible that in-flight writes exceed the max intent tracking budget (kv.transaction.max_intents_bytes). That's fine for now, but in this patch we add some observability to be aware of this happening.

I validated the new metric and log message by running:

CREATE TABLE t (k STRING PRIMARY KEY);
INSERT INTO t VALUES (RPAD('a', pow(2, 21), 'a')); // 2MB
INSERT INTO t VALUES (RPAD('b', pow(2, 21), 'b')); // 2MB
INSERT INTO t VALUES (RPAD('c', pow(2, 21), 'c')); // 2MB
BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED;
SELECT * FROM t FOR UPDATE LIMIT 5; // 6MB of locking reads, exceeding the limit of 4MB
COMMIT;
Screenshot 2024-04-30 at 8 50 09 PM
W240501 19:46:39.838255 5527 kv/kvclient/kvcoord/txn_interceptor_pipeliner.go:696 ⋮ [T2,Vdemoapp,n1,client=127.0.0.1:52087,hostssl,user=‹demo›] 731  a transaction's in-flight writes and locking reads have exceeded the intent tracking limit (kv.transaction.max_intents_bytes). in-flight writes and locking reads size: 6291483 bytes, txn: "sql txn" meta={id=a6c0b23a key=/Tenant/2/Table/112/1 iso=ReadCommitted pri=0.00869562 epo=0 ts=1714592799.819310000,0 min=1714592791.913332000,0 seq=0} lock=true stat=PENDING rts=1714592799.819310000,0 wto=false gul=1714592792.413332000,0, ba: ‹1 Scan›

Fixes: #121471

Release note: None

@miraradeva miraradeva requested a review from a team as a code owner May 1, 2024 01:08
@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented May 1, 2024

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@miraradeva miraradeva requested a review from nvb May 1, 2024 01:08
@miraradeva miraradeva added the backport-24.1.x Flags PRs that need to be backported to 24.1. label May 1, 2024
…tent budget

Since cockroachdb#121088, in-flight writes can include locking reads; because we
don't estimate the size of the locks accurately for ranged locking
reads, it is possible that in-flight writes exceed the max intent
tracking budget (`kv.transaction.max_intents_bytes`). That's fine for
now, but in this patch we add some observability to be aware of this
happening.

Fixes: cockroachdb#121471

Release note: None
Copy link
Copy Markdown
Contributor

@nvb nvb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 6 of 6 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @miraradeva)

@miraradeva
Copy link
Copy Markdown
Contributor Author

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented May 1, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-24.1.x Flags PRs that need to be backported to 24.1.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kv: add observability for exceeded maxTrackingBytes in txnPipeliner

3 participants