The `read-modify-write-in-txn` flag was added to ycsb in cockroachdb#103117. This
commit changes the default of the flag from true to false. This makes
the default configuration of ycsb more closely mirror the official ycsb
implementation, to avoid confusion when external users run the workload
and compare.

We saw in cockroachdb#103117 and in cockroachdb#107517 that this improves throughput
substantially. We will expect to see the same in roachperf once this
change is merged. After merging the PR, I will add a roachperf annotation.

Epic: None
Release note: None

105441: server: notify tenant servers of metadata changes r=yuzefovich,stevendanna a=knz

Informs (towards fixing) #98431.
Fixes #105721.
Epic: CRDB-26691

See the individual commits for details.


106325: sql/parser: parse CREATE PROCEDURE r=mgartner a=mgartner

#### sql/parser: parse CREATE PROCEDURE

`CREATE PROCEDURE` statements can now be parsed by the SQL parser.
Currently, executing `CREATE PROCEDURE` will produce an unimplemented
error.

Epic: CRDB-25388

Release note: None

#### sem/tree: rename function AST nodes

This commit renames UDF-related AST nodes with "function" in the name to
use the more general term "routine". The term "routine" is inclusive of
both UDFs and procedures (e.g., Postgres implements a `DROP ROUTINE`
statement which drops both UDFs and procedures), which is fitting
because we'll be using the same AST nodes for both `CREATE FUNCTION` and
`CREATE PROCEDURE` statements.

Release note: None

#### sem/tree: rename udf.go to create_routine.go

Release note: None


107470: server, ccl: adjust more tests to work with test tenant r=yuzefovich a=yuzefovich

Informs: #76378.
Epic: CRDB-18499.


107507:  kvserver: transfer lease when acquiring lease outside preferences r=erikgrinaker,andrewbaptist a=kvoli

First 2 commits are from #107505.

When a leaseholder is lost, any surviving replica may acquire the lease, even if it violates lease preferences. There are two main reasons for this: we need to elect a new Raft leader who will acquire the lease, which is agnostic to lease preferences, and there may not even be any surviving replicas that satisfy the lease preferences at all, so we don't want to keep the range unavailable while we try to figure this out (considering e.g. network timeouts can delay this for many seconds).

However, after acquiring a lease, we rely on the replicate queue to transfer the lease back to a replica that conforms with the preferences, which can take several minutes. In multi-region clusters, this can cause severe latency degradation if the lease is acquired in a remote region.

This PR will detect lease preference violations when a replica acquires a new lease, and eagerly enqueue it in the replicate queue for transfer (if possible). If the first process fails, the replica will be sent to purgatory and retried soon after.

Additionally, two roachtests are added for lease preference conformance timing. The first test, `.../partial-first-preference-down`, takes down one of the preferred locality nodes (1/2). The second test, `.../full-first-preference-down`, takes down both the preferred locality nodes (2/2).

Resolves #106100.
Epic: none

Release note (bug fix): When losing a leaseholder and using lease preferences, the lease can be acquired by any other replica (regardless of lease preferences) in order to restore availability as soon as possible. The new leaseholder will now immediately check if it violates the lease preferences, and attempt to transfer the lease to a replica that satisfies the preferences if possible.


107512: sql: fix crdb_internal.encode_key in some contexts r=yuzefovich a=yuzefovich

Previously, we forgot to set `CatalogBuiltins` for the internal planner which is used by `crdb_internal.encode_key` (which, in turn, is used to power `SHOW RANGE ... FOR ROW`), so evaluating it would lead to an error. For example, the internal planner is used in the backfill. This is now fixed.

Fixes: #106397.

Release note (bug fix): CockroachDB would previously return an error when using `SHOW RANGE ... FOR ROW ...` in `CREATE TABLE ... AS ...` construct, and this is now fixed.

107537: workload/ycsb: default --read-modify-write-in-txn to false r=erikgrinaker a=nvanbenschoten

The `read-modify-write-in-txn` flag was added to ycsb in #103117. This commit changes the default of the flag from true to false. This makes the default configuration of ycsb more closely mirror the official ycsb implementation, to avoid confusion when external users run the workload and compare.

We saw in #103117 and in #107517 that this improves throughput substantially. We will expect to see the same in roachperf once this change is merged. After merging the PR, I will add a roachperf annotation.

Epic: None
Release note: None

107549: persistedsqlstats: add AOST clause to statement statistics size check r=maryliag a=THardy98

Epic: None

This change as an AOST clause to the statement statistics size check.
This helps mitigate contention on the `system.statement_statistics`
table which has caused the sql stats compaction job to fail.

Release note: None

Co-authored-by: Raphael 'kena' Poss <knz@thaumogen.net>
Co-authored-by: Marcus Gartner <marcus@cockroachlabs.com>
Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
Co-authored-by: Austen McClernon <austen@cockroachlabs.com>
Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Co-authored-by: Thomas Hardy <thardy@cockroachlabs.com>

The `read-modify-write-in-txn` flag was added to ycsb in cockroachdb#103117.

This commit marks the flag as RuntimeOnly so that it doesn't break IMPORT
when a newer version of `workload` (who knows about the flag) is run against
an older version of `cockroach` (who does not).

Epic: None
Release note: None

105017: sql: add built-in functions for workload level index recommendations r=qiyanghe1998 a=qiyanghe1998

#### sql: add built-in functions for workload level index recommendations

This commit adds 4 generator built-in functions for workload level index
recommendations: workload_index_recs(), workload_index_recs(timestamptz:
TimestampTZ), workload_index_recs(budget: string) and
workload_index_recs(timestamptz: TimestampTZ, budget: string). The return types
of them are all a string column with multiple rows, one row for each index
recommendation.  The user can issue a command like `select
workload_index_recs(...);` and then get the index recommendations for the whole
workload.

Only those workload after the given timestamptz will be considered for the
index recommendations. If there is no timestamptz, all the workload stored in
`system.statement_statistics` will be taken into account. As for the budget, it
represents the space limit for the recommended indexes, e.g. 42GB, 580MiB.  If
it is specified, the workload index recommendation engine will prune the size
of all the indexes to satisfy the space constraint.

A method called workloadindexrec.FindWorkloadRecs() will be implemented to
return the array of indexRecs and error message for these 4 generator built-in
functions to invoke.

Release note: None

Epic: None

107653: cli: deflake TestDockerCLI_test_copy r=knz a=rafiss

Avoiding the line editor means that the output being parsed will be less complex.

fixes #107624
Release note: None

107726: workload/ycsb: mark --read-modify-write-in-txn flag as RuntimeOnly r=nvanbenschoten a=nvanbenschoten

The `read-modify-write-in-txn` flag was added to ycsb in #103117.

This commit marks the flag as RuntimeOnly so that it doesn't break IMPORT when a newer version of `workload` (who knows about the flag) is run against an older version of `cockroach` (who does not).

Epic: None
Release note: None

107735: authors: add sjbarag to authors r=rickystewart a=sjbarag

Release note: None
Epic: none

Co-authored-by: qiyanghe1998 <qiyang.he@cockroachlabs.com>
Co-authored-by: Rafi Shamim <rafi@cockroachlabs.com>
Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Co-authored-by: Sean Barag <barag@cockroachlabs.com>