38147: storage,roachpb: proto changes for learners r=tbg a=danhhz

Getting this in early to avoid repeatedly rebasing generated files.

Co-authored-by: Daniel Harrison <daniel.harrison@gmail.com>

In cockroachdb#38147, a new non-nullable field was added to the ReplicaDescriptor
proto that is serialized inside RangeDescriptors. RangeDescriptors are
updated using CPut to detect races. This means that if a RangeDescriptor
had been written by an old version of cockroach and we then attempt to
update it, the CPut will fail because the encoded version of it is
different (non-nullable proto2 fields are always included).

A similar issue was introduced in cockroachdb#38004 which made the StickyBit field
on RangeDescriptor non-nullable.

We could keep the fields as nullable, but this has allocation costs (and
is also more annoying to work with).

Worse, the proto spec is pretty explicit about not relying on
serialization of a given message to always produce exactly the same
bytes:

    From https://developers.google.com/protocol-buffers/docs/encoding#implications
    Do not assume the byte output of a serialized message is stable.

So instead, we've decided to stop CPut-ing protos and to change the
interface of CPut to take the expected value as bytes. To CPut a proto,
the encoded value will be read from kv, passed around with the decoded
struct, and used in the eventual CPut. This work is upcoming, but the
above two PRs are real breakages, so this commit fixes those first.

Neither of these PRs made the last alpha so no release note.

Release note: None

In cockroachdb#38147, a new non-nullable field was added to the ReplicaDescriptor
proto that is serialized inside RangeDescriptors. RangeDescriptors are
updated using CPut to detect races. This means that if a RangeDescriptor
had been written by an old version of cockroach and we then attempt to
update it, the CPut will fail because the encoded version of it is
different (non-nullable proto2 fields are always included).

A similar issue was introduced in cockroachdb#38004 which made the StickyBit field
on RangeDescriptor non-nullable.

We could keep the fields as nullable, but this has allocation costs (and
is also more annoying to work with).

Worse, the proto spec is pretty explicit about not relying on
serialization of a given message to always produce exactly the same
bytes:

    From https://developers.google.com/protocol-buffers/docs/encoding#implications
    Do not assume the byte output of a serialized message is stable.

So instead, we've decided to stop CPut-ing protos and to change the
interface of CPut to take the expected value as bytes. To CPut a proto,
the encoded value will be read from kv, passed around with the decoded
struct, and used in the eventual CPut. This work is upcoming, but the
above two PRs are real breakages, so this commit fixes those first.

Neither of these PRs made the last alpha so no release note.

Touches cockroachdb#38308

Release note: None

In cockroachdb#38147, a new non-nullable field was added to the ReplicaDescriptor
proto that is serialized inside RangeDescriptors. RangeDescriptors are
updated using CPut to detect races. This means that if a RangeDescriptor
had been written by an old version of cockroach and we then attempt to
update it, the CPut will fail because the encoded version of it is
different (non-nullable proto2 fields are always included).

A similar issue was introduced in cockroachdb#38004 which made the StickyBit field
on RangeDescriptor non-nullable.

We could keep the fields as nullable, but this has allocation costs (and
is also more annoying to work with).

Worse, the proto spec is pretty explicit about not relying on
serialization of a given message to always produce exactly the same
bytes:

    From https://developers.google.com/protocol-buffers/docs/encoding#implications
    Do not assume the byte output of a serialized message is stable.

So instead, we've decided to stop CPut-ing protos and to change the
interface of CPut to take the expected value as bytes. To CPut a proto,
the encoded value will be read from kv, passed around with the decoded
struct, and used in the eventual CPut. This work is upcoming, but the
above two PRs are real breakages, so this commit fixes those first.

Neither of these PRs made the last alpha so no release note.

Touches cockroachdb#38308

Release note: None

38302: storage: CPut bytes instead of a proto when updating RangeDescriptors r=tbg a=danhhz

In #38147, a new non-nullable field was added to the ReplicaDescriptor
proto that is serialized inside RangeDescriptors. RangeDescriptors are
updated using CPut to detect races. This means that if a RangeDescriptor
had been written by an old version of cockroach and we then attempt to
update it, the CPut will fail because the encoded version of it is
different (non-nullable proto2 fields are always included).

A similar issue was introduced in #38004 which made the StickyBit field
on RangeDescriptor non-nullable.

We could keep the fields as nullable, but this has allocation costs (and
is also more annoying to work with).

Worse, the proto spec is pretty explicit about not relying on
serialization of a given message to always produce exactly the same
bytes:

    From https://developers.google.com/protocol-buffers/docs/encoding#implications
    Do not assume the byte output of a serialized message is stable.

So instead, we've decided to stop CPut-ing protos and to change the
interface of CPut to take the expected value as bytes. To CPut a proto,
the encoded value will be read from kv, passed around with the decoded
struct, and used in the eventual CPut. This work is upcoming, but the
above two PRs are real breakages, so this commit fixes those first.

Neither of these PRs made the last alpha so no release note.

Touches #38308
Closes #38183

Release note: None

Co-authored-by: Daniel Harrison <daniel.harrison@gmail.com>

As seen in cockroachdb#38004 and cockroachdb#38147, this first came up in the context of CPut
with a proto expected value. If the serialization didn't exactly match,
the CPut would get a condition failed error, even if the two protos were
logically equivalent (missing field vs field with default value). We
were hitting this in a mixed version cluster when trying to add a
non-nullable field to RangeDescriptor, which is updated via CPut to
detect update races. The new fields ended up having to be nullable,
which has allocation consequences as well as ergonomic ones.

In cockroachdb#38302, we changed the RangeDescriptor CPut to use a *roachpb.Value
exactly as it was read from KV for the expected value, but no other
callsites were updated. To prevent future issues, this commit changes
the CPut signature to accept a *roachpb.Value for the expected value
instead of an interface{}.

The old method signature is left as CPutDeprecated for now. The CPut
usages that are trivial to switch are switched and ones that require any
thought are left for a followup PR.

Release note: None

39714: client: force CPut callers to use a *roachpb.Value expected value r=tbg a=danhhz

As seen in #38004 and #38147, this first came up in the context of CPut
with a proto expected value. If the serialization didn't exactly match,
the CPut would get a condition failed error, even if the two protos were
logically equivalent (missing field vs field with default value). We
were hitting this in a mixed version cluster when trying to add a
non-nullable field to RangeDescriptor, which is updated via CPut to
detect update races. The new fields ended up having to be nullable,
which has allocation consequences as well as ergonomic ones.

In #38302, we changed the RangeDescriptor CPut to use a *roachpb.Value
exactly as it was read from KV for the expected value, but no other
callsites were updated. To prevent future issues, this commit changes
the CPut signature to accept a *roachpb.Value for the expected value
instead of an interface{}.

The old method signature is left as CPutDeprecated for now. The CPut
usages that are trivial to switch are switched and ones that require any
thought are left for a followup PR.

Release note: None

Co-authored-by: Daniel Harrison <daniel.harrison@gmail.com>