Skip to content

kv/kvnemesis: TestKVNemesisMultiNode failed #101048

@cockroach-teamcity

Description

@cockroach-teamcity

kv/kvnemesis.TestKVNemesisMultiNode failed with artifacts on release-22.2 @ 9a7c644e565b21d29db26a0a82524a00809d0a8c:

          | 	github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_send.go:938
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).SendWithWriteBytes
          | 	github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_send.go:183
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).SendWithWriteBytes
          | 	github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/store_send.go:205
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).SendWithWriteBytes
          | 	github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/stores.go:203
          | github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal
          | 	github.com/cockroachdb/cockroach/pkg/server/node.go:1102
          | github.com/cockroachdb/cockroach/pkg/server.(*Node).Batch
          | 	github.com/cockroachdb/cockroach/pkg/server/node.go:1155
          | github.com/cockroachdb/cockroach/pkg/roachpb._Internal_Batch_Handler.func1
          | 	github.com/cockroachdb/cockroach/pkg/roachpb/bazel-out/k8-fastbuild/bin/pkg/roachpb/roachpb_go_proto_/github.com/cockroachdb/cockroach/pkg/roachpb/api.pb.go:9619
          | github.com/cockroachdb/cockroach/pkg/util/tracing/grpcinterceptor.ServerInterceptor.func1
          | 	github.com/cockroachdb/cockroach/pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:96
          | google.golang.org/grpc.chainUnaryInterceptors.func1.1
          | 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:1117
          | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func3
          | 	github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:263
          | google.golang.org/grpc.chainUnaryInterceptors.func1.1
          | 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:1120
          | github.com/cockroachdb/cockroach/pkg/rpc.kvAuth.unaryInterceptor
          | 	github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/auth.go:73
          | google.golang.org/grpc.chainUnaryInterceptors.func1.1
          | 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:1120
          | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1.1
          | 	github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:232
          | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr
          | 	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:341
          | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1
          | 	github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:230
          | google.golang.org/grpc.chainUnaryInterceptors.func1.1
          | 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:1120
          | google.golang.org/grpc.chainUnaryInterceptors.func1
          | 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:1122
          | github.com/cockroachdb/cockroach/pkg/roachpb._Internal_Batch_Handler
          | 	github.com/cockroachdb/cockroach/pkg/roachpb/bazel-out/k8-fastbuild/bin/pkg/roachpb/roachpb_go_proto_/github.com/cockroachdb/cockroach/pkg/roachpb/api.pb.go:9621
          | google.golang.org/grpc.(*Server).processUnaryRPC
          | 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:1283
          | google.golang.org/grpc.(*Server).handleStream
          | 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:1620
          | google.golang.org/grpc.(*Server).serveStreams.func1.2
          | 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:922
          | runtime.goexit
          | 	GOROOT/src/runtime/asm_amd64.s:1594
        Wraps: (4) unable to find store 2 in range r59:/Table/100/"{1239343b"-2fb2ccb3"} [(n4,s4):1, (n1,s1):2, next=5, gen=31, sticky=9223372036.854775807,2147483647]
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *errbase.opaqueWrapper (4) *errutil.leafError
    kvnemesis_test.go:113: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/1f42cf5be2fc021646bf9b2daf5eaef3/logTestKVNemesisMultiNode3020963917
--- FAIL: TestKVNemesisMultiNode (22.87s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

/cc @cockroachdb/kv

This test on roachdash | Improve this report!

Jira issue: CRDB-26778

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-test-failureBroken test (automatically or manually discovered).O-robotOriginated from a bot.T-kvKV TeamX-stalebranch-release-22.2Used to mark GA and release blockers, technical advisories, and bugs for 22.2no-test-failure-activity

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions