The cause of the get-user-timeout errors is unknown. Part of the problem is that the stack trace gets cut off at

  |   | github.com/cockroachdb/cockroach/pkg/sql.retrieveSessionInitInfoWithCache
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/user.go:238

which does not explain what is actually being blocked.

The reason that the stack trace is cut off is that the timeout is initiated by contextutil.RunWithTimeout which results in a "simple" (no stack trace) context.DeadlineExceeded error.

retrieveSessionInitInfoWithCache is the first line in the stack trace because it calls errors.Wrap on the error from context.DeadlineExceeded.

To get fuller stack trace, context.DeadlineExceeded must be wrapped immediately (errors.Wrap or errors.WithStack) before it bubbles up.

This is to diagnose the get-user-timeout errors here where the stack trace is cut off before the place we call errors.Wrap(...): https://github.com/cockroachlabs/support/issues/2017#issuecomment-1402418577

Jira issue: CRDB-23745