Skip to content

server: /data-distribution page returns error after creating tenant #97993

Closed
#99142
Closed
server: /data-distribution page returns error after creating tenant#97993
#99142
Assignees
zachlite
Labels
C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.GA-blockerbranch-release-23.1Used to mark GA and release blockers, technical advisories, and bugs for 23.1
@dhartunian

Description

@dhartunian
dhartunian
opened on Mar 3, 2023

Describe the problem

After an application tenant is created, the "Data Distribution and Zone Configs" page no longer

To Reproduce

Start a cluster, built off of master:

./cockroach start-single-node --certs-dir=./certs

Create user:

create user david with password 'cockroach';
grant admin to david

Login to DB console, go to https://127.0.0.1:8080/#/data-distribution. It should work

Run SQL:

create tenant 'app';

Refresh data distribution. Error is returned.

Screenshot 2023-03-03 at 2 43 38 PM

Expected behavior
Working data distribution page

Additional data / screenshots

E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266  invalid key prefix: /Min
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +(1) attached stack trace
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  -- stack trace:
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/keys.sqlDecoder.DecodeTablePrefix
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/keys/sql.go:204
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/server.(*adminServer).dataDistributionHelper.func2
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/server/admin.go:3075
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/kv.runTxn.func1
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/kv/db.go:965
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/kv.(*Txn).exec
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/kv/txn.go:928
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/kv.runTxn
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/kv/db.go:964
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/kv.(*DB).TxnWithAdmissionControl
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/kv/db.go:927
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/kv.(*DB).Txn
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/kv/db.go:902
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/server.(*adminServer).dataDistributionHelper
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/server/admin.go:3036
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/server.(*adminServer).DataDistribution
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/server/admin.go:2927
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/server/serverpb._Admin_DataDistribution_Handler.func1
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/server/serverpb/bazel-out/darwin_arm64-fastbuild/bin/pkg/server/serverpb/serverpb_go_proto_/github.com/cockroachdb/cockroach/pkg/server/serverpb/admin.pb.go:6268
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/util/tracing/grpcinterceptor.ServerInterceptor.func1
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:115
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | google.golang.org/grpc.chainUnaryInterceptors.func1.1
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       google.golang.org/grpc/external/org_golang_google_grpc/server.go:1162
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func3
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:266
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | google.golang.org/grpc.chainUnaryInterceptors.func1.1
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       google.golang.org/grpc/external/org_golang_google_grpc/server.go:1165
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/rpc.kvAuth.unaryInterceptor
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/auth.go:92
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | google.golang.org/grpc.chainUnaryInterceptors.func1.1
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       google.golang.org/grpc/external/org_golang_google_grpc/server.go:1165
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1.1
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:233
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:322
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:231
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | google.golang.org/grpc.chainUnaryInterceptors.func1.1
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       google.golang.org/grpc/external/org_golang_google_grpc/server.go:1165
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | google.golang.org/grpc.chainUnaryInterceptors.func1
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       google.golang.org/grpc/external/org_golang_google_grpc/server.go:1167
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | github.com/cockroachdb/cockroach/pkg/server/serverpb._Admin_DataDistribution_Handler
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       github.com/cockroachdb/cockroach/pkg/server/serverpb/bazel-out/darwin_arm64-fastbuild/bin/pkg/server/serverpb/serverpb_go_proto_/github.com/cockroachdb/cockroach/pkg/server/serverpb/admin.pb.go:6270
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | google.golang.org/grpc.(*Server).processUnaryRPC
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       google.golang.org/grpc/external/org_golang_google_grpc/server.go:1340
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | google.golang.org/grpc.(*Server).handleStream
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       google.golang.org/grpc/external/org_golang_google_grpc/server.go:1713
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | google.golang.org/grpc.(*Server).serveStreams.func1.2
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       google.golang.org/grpc/external/org_golang_google_grpc/server.go:965
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  | runtime.goexit
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +  |       GOROOT/src/runtime/asm_arm64.s:1172
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +Wraps: (2) invalid key prefix: /Min
E230303 19:43:23.608397 7498 server/admin.go:2929 ⋮ [T1] 266 +Error types: (1) *withstack.withStack (2) *errutil.leafError

Environment:

  • CockroachDB version: 23.1.0
  • Client app: DB Console

Discovered while investigating #97942. Ideally the solution here works for tenants as well.

Jira issue: CRDB-25007

Metadata

Metadata

Assignees

Labels

C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.GA-blockerbranch-release-23.1Used to mark GA and release blockers, technical advisories, and bugs for 23.1

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions