ccl/backupccl: TestDataDriven failed #96149
Description
ccl/backupccl.TestDataDriven failed with artifacts on master @ 69dd453d0e61e258f402c5751de310405743cd18:
Fatal error:
panic: for 3 in test_certs: tenant client cert not found [recovered]
panic: for 3 in test_certs: tenant client cert not found
Stack:
goroutine 543259385 [running]:
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).recover(0x3fe8020?, {0x6e7c438, 0xc070408990})
github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:229 +0x6a
panic({0x4fccb00, 0xc00790bf00})
GOROOT/src/runtime/panic.go:884 +0x212
github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx(0xc04581a000, {0xc01afac4c0, 0x1, 0x1?})
github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:213 +0x10de
github.com/cockroachdb/cockroach/pkg/server.newGRPCServer(0x6e7c438?)
github.com/cockroachdb/cockroach/pkg/server/grpc_server.go:36 +0xaa
github.com/cockroachdb/cockroach/pkg/server.makeTenantSQLServerArgs({_, _}, _, {0xc01e744000, 0xc0dea2a200, 0xc010224620, 0xc038830eb0, 0xc017e8b300, 0xc01a395ce0, {0xc010224620, ...}, ...}, ...)
github.com/cockroachdb/cockroach/pkg/server/tenant.go:960 +0x1686
github.com/cockroachdb/cockroach/pkg/server.NewTenantServer({_, _}, _, {0xc01e744000, 0xc0dea2a200, 0xc010224620, 0xc038830eb0, 0xc017e8b300, 0xc01a395ce0, {0xc010224620, ...}, ...}, ...)
github.com/cockroachdb/cockroach/pkg/server/tenant.go:172 +0x165
github.com/cockroachdb/cockroach/pkg/server.(*Server).startTenantServerInternal(_, {_, _}, {0xc01e744000, 0xc0dea2a200, 0xc010224620, 0xc038830eb0, 0xc017e8b300, 0xc01a395ce0, {0xc010224620, ...}, ...}, ...)
github.com/cockroachdb/cockroach/pkg/server/server_controller_new_server.go:148 +0x21b
github.com/cockroachdb/cockroach/pkg/server.(*Server).newTenantServer(_, {_, _}, _, _, {{0x0, 0x0}, {0x0}, {{0x0, 0x0}, ...}, ...})
github.com/cockroachdb/cockroach/pkg/server/server_controller_new_server.go:82 +0x2fb
github.com/cockroachdb/cockroach/pkg/server.(*serverController).startServerInternal(0xc0380cd140, {0x6e7c438, 0xc070408990}, 0xc040349f30)
github.com/cockroachdb/cockroach/pkg/server/server_controller_orchestration.go:361 +0x196
github.com/cockroachdb/cockroach/pkg/server.(*serverController).startControlledServer.func2({0x6e7c438, 0xc070408990})
github.com/cockroachdb/cockroach/pkg/server/server_controller_orchestration.go:233 +0x296
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470 +0x146
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:461 +0x43b
Log preceding fatal error
----
2 newname 2
datadriven_test.go:411:
/home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/4365/execroot/com_github_cockroachdb_cockroach/bazel-out/k8-fastbuild/bin/pkg/ccl/backupccl/backupccl_test_/backupccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/backupccl/testdata/backup-restore/restore-tenants:125:
exec-sql [0 args]
BACKUP TENANT 2 INTO 'nodelocal://1/tenant2'
----
datadriven_test.go:411:
/home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/4365/execroot/com_github_cockroachdb_cockroach/bazel-out/k8-fastbuild/bin/pkg/ccl/backupccl/backupccl_test_/backupccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/backupccl/testdata/backup-restore/restore-tenants:129:
exec-sql [0 args]
RESTORE TENANT 2 FROM LATEST IN 'nodelocal://1/tenant2' WITH tenant_name = 'another-name';
----
datadriven_test.go:411:
/home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/4365/execroot/com_github_cockroachdb_cockroach/bazel-out/k8-fastbuild/bin/pkg/ccl/backupccl/backupccl_test_/backupccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/backupccl/testdata/backup-restore/restore-tenants:133:
query-sql [0 args]
SELECT id,name,service_mode FROM system.tenants WHERE name = 'another-name';
----
3 another-name 2
testutils.go:199: no Invalid Descriptors
testutils.go:199: no Invalid Descriptors
*
* ERROR: a panic has occurred!
* for 3 in test_certs: tenant client cert not found
* (1) attached stack trace
* -- stack trace:
* | runtime.gopanic
* | GOROOT/src/runtime/panic.go:884
* | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx
* | github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:213
* | github.com/cockroachdb/cockroach/pkg/server.newGRPCServer
* | github.com/cockroachdb/cockroach/pkg/server/grpc_server.go:36
* | github.com/cockroachdb/cockroach/pkg/server.makeTenantSQLServerArgs
* | github.com/cockroachdb/cockroach/pkg/server/tenant.go:960
* | github.com/cockroachdb/cockroach/pkg/server.NewTenantServer
* | github.com/cockroachdb/cockroach/pkg/server/tenant.go:172
* | github.com/cockroachdb/cockroach/pkg/server.(*Server).startTenantServerInternal
* | github.com/cockroachdb/cockroach/pkg/server/server_controller_new_server.go:148
* | github.com/cockroachdb/cockroach/pkg/server.(*Server).newTenantServer
* | github.com/cockroachdb/cockroach/pkg/server/server_controller_new_server.go:82
* | github.com/cockroachdb/cockroach/pkg/server.(*serverController).startServerInternal
* | github.com/cockroachdb/cockroach/pkg/server/server_controller_orchestration.go:361
* | github.com/cockroachdb/cockroach/pkg/server.(*serverController).startControlledServer.func2
* | github.com/cockroachdb/cockroach/pkg/server/server_controller_orchestration.go:233
* | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
* | github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470
* | runtime.goexit
* | GOROOT/src/runtime/asm_amd64.s:1594
* Wraps: (2) for 3 in test_certs: tenant client cert not found
* Error types: (1) *withstack.withStack (2) *security.Error
*
Parameters: TAGS=bazel,gss,deadlock
This test on roachdash | Improve this report!
Jira issue: CRDB-23967