Skip to content

kvserver: default admission.kv.pause_replication_io_threshold to 0.8#86147

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
tbg:pausing-on-default
Aug 15, 2022
Merged

kvserver: default admission.kv.pause_replication_io_threshold to 0.8#86147
craig[bot] merged 1 commit intocockroachdb:masterfrom
tbg:pausing-on-default

Conversation

@tbg
Copy link
Copy Markdown
Member

@tbg tbg commented Aug 15, 2022

Closes #83920.

Release note (ops change): The
admission.kv.pause_replication_threshold cluster setting is now set to
a default value of 0.8. On a fully migrated v22.2+ deployment, this will
allow the KV layer to pause replication streams to followers located on
stores that are close to activating their I/O admission control
subsystem (thereby protecting these followers from additional overload).
The cluster setting can be disabled by setting to zero.

Release justification: enables new functionality

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@tbg tbg requested review from a team and erikgrinaker August 15, 2022 15:15
@tbg tbg marked this pull request as ready for review August 15, 2022 15:15
@tbg tbg requested a review from a team as a code owner August 15, 2022 15:15
@tbg
Copy link
Copy Markdown
Member Author

tbg commented Aug 15, 2022

going to wait for #85739

Copy link
Copy Markdown
Contributor

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cluster setting can be disabled by setting a high value of, say, 10000.

The description says "zero to disable". Is this no longer accurate? I'd intuitively assume that was the case.

Closes cockroachdb#83920.

Release note (ops change): The
`admission.kv.pause_replication_threshold` cluster setting is now set to
a default value of 0.8. On a fully migrated v22.2+ deployment, this will
allow the KV layer to pause replication streams to followers located on
stores that are close to activating their I/O admission control
subsystem (thereby protecting these followers from additional overload).
The cluster setting can be disabled by setting it to zero.
@tbg tbg force-pushed the pausing-on-default branch from 654e314 to 1fa00d4 Compare August 15, 2022 18:25
@tbg
Copy link
Copy Markdown
Member Author

tbg commented Aug 15, 2022

That just changed in #85739, but I didn't update the description. I did that because (at least at some point) the cluster setting was read in two places and it was bug prone to have to special case zero. But now we only query it in one place; brought back the special casing and adjusted the release note.

bors r=erikgrinaker

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 15, 2022

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 15, 2022

Build succeeded:

Copy link
Copy Markdown
Contributor

@irfansharif irfansharif left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The release note has a typo in the cluster setting that might make it to public docs. It needs to be admission.kv.pause_replication_io_threshold.

tbg added a commit to tbg/cockroach that referenced this pull request Aug 24, 2022
(Morally) reverts cockroachdb#86147. We'll roll this setting out over time.

See: cockroachdb#86775

Release justification: more conservative choice for a new cluster setting
Release note (ops change): the
admission.kv.pause_replication_io_threshold cluster setting now default
to zero (off). This supersedes an earlier release note about this
setting.
craig bot pushed a commit that referenced this pull request Aug 25, 2022
86625: roachtest: compile cockroach-short with --crdb_test and use in sqlsmith r=yuzefovich a=yuzefovich

This commit adds another step to the nightly roachtest invocation to
compile `cockroach-short` binary with `--crdb_test` build tag. Then it
also adds plumbing throughout the roachtest infrastructure to expose
that newly-compiled binary which is now used in 50% cases in the sqlsmith
roachtest.

Fixes: #83186.

Release justification: testing-only change.

Release note: None

86675: builtins: fix json_build_object for enums and void in the key r=yuzefovich a=yuzefovich

Previously, we would run into an internal error with `json_build_object`
builtin if an enum or void was passed for the key part, and this is now
fixed. This commit also sorts all datums lexicographically to make it
easier to see whether all scalar datums are mentioned.

Fixes: #84368.

Release justification: bug fix.

Release note (bug fix): Previously, CockroachDB would return an internal
error when evaluating `json_build_object` builtin when an enum or a void
datums were passed as the first argument, and this is now fixed.

86759: storage: implement `String()` for `MVCCRangeKeyStack` r=jbowens a=erikgrinaker

Release justification: bug fixes and low-risk updates to new functionality

Release note: None

86776: kvserver: disable replica pausing by default r=erikgrinaker a=tbg

(Morally) reverts #86147. We'll roll this setting out over time.

See: #86775

Release justification: more conservative choice for a new cluster setting
Release note (ops change): the
admission.kv.pause_replication_io_threshold cluster setting now default
to zero (off). This supersedes an earlier release note about this
setting.


Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
Co-authored-by: Erik Grinaker <grinaker@cockroachlabs.com>
Co-authored-by: Tobias Grieger <tobias.b.grieger@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kvserver: default admission.kv.pause_replication_io_threshold to a nonzero value

4 participants