Skip to content

kvevent: Fix bookkeeping related to out of quota handling#87997

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
miretskiy:flake
Sep 16, 2022
Merged

kvevent: Fix bookkeeping related to out of quota handling#87997
craig[bot] merged 1 commit intocockroachdb:masterfrom
miretskiy:flake

Conversation

@miretskiy
Copy link
Copy Markdown
Contributor

Fix a bug in blocking buffer bookkeeping which could cause an out of quota event to not be delivered when there is a single consumer.

Fixes #87828

Release note: None
Release justification: bug fix

@miretskiy miretskiy requested a review from a team as a code owner September 15, 2022 14:29
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Contributor

@HonoreDB HonoreDB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 4 files at r1.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ajwerner, @jayshrivastava, and @miretskiy)


pkg/ccl/changefeedccl/kvevent/blocking_buffer.go line 181 at r1 (raw file):

	}
	if b.mu.numBlocked == 0 {
		b.mu.canFlush = false

Can you say more about why canFlush is set to false here? (Would needFlush be a more accurate name?)

@miretskiy
Copy link
Copy Markdown
Contributor Author

Can you say more about why canFlush is set to false here? (Would needFlush be a more accurate name?)

Done; And no, I think canFlush is more accurate than needFlush. The fact that we are blocked doesn't mean
that we need to flush -- flush can happen anyway. What that means is that if we truly are stuck -- namely if this
buffer has nothing buffered, and we are blocked, waiting for memory (meaning, there is nothing to release by this buffer), then the buffer can issue flush request in the hopes of releasing memory.

@miretskiy miretskiy force-pushed the flake branch 2 times, most recently from a9bbc8f to 7962533 Compare September 16, 2022 13:20
Fix a bug in blocking buffer bookkeeping which could
cause an out of quota event to not be delivered when
there is a single consumer.

In addition, fix a bug where FlushEvents were counted
toward quota.  This was wrong since FlushEvents are meant
to signal that we are out of quota, and blocking to acquire
more quota for those events is wrong.
This issue was manifested by `TestChangefeedFlushesSinkToReleaseMemory`
test taking excessive long time.

Fixes cockroachdb#87828

Release note: None
Release justification: bug fix
Copy link
Copy Markdown
Contributor

@HonoreDB HonoreDB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ajwerner)

@miretskiy
Copy link
Copy Markdown
Contributor Author

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Sep 16, 2022

Build succeeded:

@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented Sep 16, 2022

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from 39055e6 to blathers/backport-release-22.1-87997: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 22.1.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

changefeedccl: TestBlockingBufferNotifiesConsumerWhenOutOfMemory is flakey

4 participants