Describe the problem

We don't have great guidelines/infra for runtime assertions. Some of us have, in previous jobs, worked on products that relied heavily on runtime assertions (that would be compiled out in the production builds) and have found this very useful.

Currently, there are a few issues:

1. there isn't a great way to even write assertions succinctly. A require-style API would be great, but instead you have to do something like

if buildutils.CrdbTestBuild {
  if fooCondition != barCondition {
    log.Fatalf(ctx, "%s", errors.AssertionFailedf("Halp! fooCondition was %v!", barCondition)
  }
}

That's pretty noisy and tends to pollute the code, and also it's just long so people don't type it when they should.

2. they're not enabled during nightly roachtests. This is because we only have this one class of assertions, and lots of very slow stuff might be gated behind it. We don't have a good separation between "cheap" and "expensive" assertions, and we don't have a clear distinction between "performance" and "correctness" roachtests at the moment.

#94979 has some links to ideas.

Jira issue: CRDB-23260