Skip to content

ccl/kvccl/kvtenantccl: TestConnectorRetriesUnreachable failed #67246

@cockroach-teamcity

Description

@cockroach-teamcity

ccl/kvccl/kvtenantccl.TestConnectorRetriesUnreachable failed with artifacts on master @ 84ec89c77841016da0b9c4c71772a4304bad45a5:

  github.com/cockroachdb/cockroach/pkg/rpc.(*Context).grpcDialNodeInternal.func1.1()
      /go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:1061 +0x94
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1()
      /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:347 +0x149

Goroutine 106 (running) created at:
  google.golang.org/grpc/internal/transport.newHTTP2Client()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x1488
  google.golang.org/grpc/internal/transport.NewClientTransport()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/transport.go:580 +0x6c7
  google.golang.org/grpc.(*addrConn).createTransport()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:1323 +0x4f7
  google.golang.org/grpc.(*addrConn).tryAllAddrs()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:1253 +0x507
  google.golang.org/grpc.(*addrConn).resetTransport()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:1168 +0x33b

Goroutine 103 (running) created at:
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask()
      /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:342 +0x107
  github.com/cockroachdb/cockroach/pkg/rpc.(*Context).grpcDialNodeInternal.func1()
      /go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:1059 +0x411
  sync.(*Once).doSlow()
      /usr/local/go/src/sync/once.go:68 +0x109
  sync.(*Once).Do()
      /usr/local/go/src/sync/once.go:59 +0x68
  github.com/cockroachdb/cockroach/pkg/rpc.(*Context).grpcDialNodeInternal()
      /go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:1053 +0x25c
  github.com/cockroachdb/cockroach/pkg/rpc.(*Context).GRPCUnvalidatedDial()
      /go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:1002 +0xaa
  github.com/cockroachdb/cockroach/pkg/ccl/kvccl/kvtenantccl.(*Connector).dialAddr.func1()
      /go/src/github.com/cockroachdb/cockroach/pkg/ccl/kvccl/kvtenantccl/connector.go:420 +0x45
  github.com/cockroachdb/cockroach/pkg/util/contextutil.RunWithTimeout()
      /go/src/github.com/cockroachdb/cockroach/pkg/util/contextutil/context.go:89 +0xed
  github.com/cockroachdb/cockroach/pkg/ccl/kvccl/kvtenantccl.(*Connector).dialAddr()
      /go/src/github.com/cockroachdb/cockroach/pkg/ccl/kvccl/kvtenantccl/connector.go:419 +0x1d5
  github.com/cockroachdb/cockroach/pkg/ccl/kvccl/kvtenantccl.(*Connector).dialAddrs()
      /go/src/github.com/cockroachdb/cockroach/pkg/ccl/kvccl/kvtenantccl/connector.go:404 +0x347
  github.com/cockroachdb/cockroach/pkg/ccl/kvccl/kvtenantccl.(*Connector).getClient.func1.1()
      /go/src/github.com/cockroachdb/cockroach/pkg/ccl/kvccl/kvtenantccl/connector.go:372 +0x64
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr()
      /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:326 +0x129
  github.com/cockroachdb/cockroach/pkg/ccl/kvccl/kvtenantccl.(*Connector).getClient.func1()
      /go/src/github.com/cockroachdb/cockroach/pkg/ccl/kvccl/kvtenantccl/connector.go:369 +0x25a
  github.com/cockroachdb/cockroach/pkg/util/syncutil/singleflight.(*Group).doCall()
      /go/src/github.com/cockroachdb/cockroach/pkg/util/syncutil/singleflight/singleflight.go:128 +0x51
==================
W210706 05:20:51.536642 132 ccl/kvccl/kvtenantccl/connector.go:159  [tenant-connector] 8  error consuming GossipSubscription RPC: rpc error: code = Canceled desc = context canceled
    testing.go:1092: race detected during execution of test
--- FAIL: TestConnectorRetriesUnreachable (0.04s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestConnectorRetriesUnreachable PKG=./pkg/ccl/kvccl/kvtenantccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Parameters in this failure:

  • GOFLAGS=-race -parallel=4

/cc @cockroachdb/kv nvanbenschoten

This test on roachdash | Improve this report!

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-test-failureBroken test (automatically or manually discovered).O-robotOriginated from a bot.branch-masterFailures and bugs on the master branch.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions