`TxnCoordSender` made sure to track and eventually clean up intents
for transactions that were put into Writing mode before seeing an
error. However, it did not previously make an effort to track and
clean up intents when it saw an `AmbiguousResultErrors` during the
first request of a transaction. This change fixes that.

This was giving me all kinds of trouble when trying to write a test
for cockroachdb#20589. Split requests continued to leave lots of dangling intents
around whenever they were canceled, which slowed down the test and
required lots of intent pushing. This change seems to resolve those
complications.

Release note (bug fix): clean up dangling intents eagerly when
AmbiguousResultErrors are seen.

`TxnCoordSender` made sure to track and eventually clean up intents
for transactions that were put into Writing mode before seeing an
error. However, it did not previously make an effort to track and
clean up intents when it saw an `AmbiguousResultErrors` during the
first request of a transaction. This change fixes that.

This was giving me all kinds of trouble when trying to write a test
for cockroachdb#20589. Split requests continued to leave lots of dangling intents
around whenever they were canceled, which slowed down the test and
required lots of intent pushing. This change seems to resolve those
complications.

Release note (bug fix): clean up dangling intents eagerly when
AmbiguousResultErrors are seen.

Testing has demonstrated that hotspot workloads which write large amounts
of data to a small range of keys can overload the `splitQueue` and create
excessively large ranges. Previous efforts to parallelize replica splits
have helped improve the queue's ability to keep up with these workloads,
but speeding up splits alone will never be able to fully prevent unbounded
range growth. Large ranges (those in the GB range) are problematic for
multiple reasons including that they slow down snapshots, consistency checks,
and other processes that scan over entire ranges and haven't been built with
these size ranges in mind. In the past we've attempted to prevent these
processes from running when ranges get too large, but this has created
other issues like cockroachdb#20589.

This change introduces a proactive backpressure mechanism to delay writes
once a range gets too large while we wait for its split to succeed. This
accomplishes the task of preventing ranges from getting to large because
it stops the range from growing and prevents new writes from getting in
the way of the split attempt. This has been shown to create an effective
soft limit on the maximum size of the range. By default, this limit is set
to twice the `range_max_bytes` setting, but this change also introduces an
environment variable for configuring the limit.

Release note (performance improvement): Writes will now be backpressured
when ranges grow to large until the range is successfully able to split.
This prevents unbounded range growth and improves a clusters ability to
stay healthy under hotspot workloads.

Testing has demonstrated that hotspot workloads which write large amounts
of data to a small range of keys can overload the `splitQueue` and create
excessively large ranges. Previous efforts to parallelize replica splits
have helped improve the queue's ability to keep up with these workloads,
but speeding up splits alone will never be able to fully prevent unbounded
range growth. Large ranges (those in the GB range) are problematic for
multiple reasons including that they slow down snapshots, consistency checks,
and other processes that scan over entire ranges and haven't been built with
these size ranges in mind. In the past we've attempted to prevent these
processes from running when ranges get too large, but this has created
other issues like cockroachdb#20589.

This change introduces a proactive backpressure mechanism to delay writes
once a range gets too large while we wait for its split to succeed. This
accomplishes the task of preventing ranges from getting to large because
it stops the range from growing and prevents new writes from getting in
the way of the split attempt. This has been shown to create an effective
soft limit on the maximum size of the range. By default, this limit is set
to twice the `range_max_bytes` setting, but this change also introduces an
environment variable for configuring the limit.

Release note (performance improvement): Writes will now be backpressured
when ranges grow to large until the range is successfully able to split.
This prevents unbounded range growth and improves a clusters ability to
stay healthy under hotspot workloads.

A simple data generator that makes a single large range. The created dataset can
then be used for various tests, for example to exercises issues as the ones
ultimately leading to cockroachdb#20589, or to
make sure [large snapshots] work (once implemented).

This is a work-in-progress because I haven't reached clarity on what the best
way to hook things up in these tests would be. Do we want to create the datasets
and upload them somewhere? That has been fragile in the past as the upload
progress usually gets seldom exercised and thus rots. The alternative (which I'm
leaning towards) is to bundle this binary with the test code (either explicitly
or via use as a library) and create fresh test data every time (these tests
would run as nightlies and so dataset generation speed isn't the top concern).

In making these decisions, we should also take into account more involved datasets
that can't as easily be generated from a running cluster, such as [gcpressurizer].

For those, my current take is that we'll just generate an initialized data dir,
open the resulting RocksDB instance manually again, and write straight into it
(via some facility that updates stats correctly, i.e. presumably `MVCCPut` and
friends).

Release note: None

[large snapshots]: cockroachdb#16954
[gcpressurizer]: cockroachdb#18661