Skip to content

sql: audit usages of tenantPrefix.PrefixEnd() and codec.TenantSpan() #104928

@arulajmani

Description

@arulajmani

Describe the problem

Recent investigations have revealed that #104606 is caused by split points created during tenant creation, which are generated in:

tenantStartKey := roachpb.RKey(ms.codec.TenantPrefix())
tenantEndKey := tenantStartKey.PrefixEnd()
splits = []roachpb.RKey{tenantStartKey, tenantEndKey}

The problem here is the split point that corresponds to tenantEndKey. Normally, there is no difference between MakeTenantPrefix(ID).PrefixEnd() and MakeTenantPrefix(ID + 1). However, this doesn't hold true at the boundary points of EncodeUvarintAscending -- @nvanbenschoten demonstrated this over at: https://go.dev/play/p/1KKbBMwENyX

Interestingly, because these two keys aren't the same at certain ID values, it means that if the key was generated using MakeTenantPrefix(ID).PrefixEnd(), a tenant ID cannot be parsed from it. That's what caused the fatals at the linked issue above.

All this points to a misunderstanding, in various places in the code, where we aren't being intentional about which one we need or do not need. We should audit all places in the code that use codec.TenantSpan() and <tenant_prefix>.PrefixEnd() to ensure we're doing the right thing (given our new found understanding of the subtleties here).

cc @stevendanna @knz @ecwall

Jira issue: CRDB-28780
Epic: CRDB-26091

Metadata

Metadata

Assignees

Labels

C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.O-supportWould prevent or help troubleshoot a customer escalation - bugs, missing observability/tooling, docsT-multitenantIssues owned by the multi-tenant virtual team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions