Skip to content

server: datarace in TestListSessionsV2 #67237

@arulajmani

Description

@arulajmani

Describe the problem

Build: https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_MergedExtendedCi/3149479?

There seems to be a datarace in TestListSessionsV2. Stack trace from the build above:

=== RUN   TestReplicaGCQueueDropReplicaDirect
    test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestReplicaGCQueueDropReplicaDirect442367224
    test_log_scope.go:74: use -show-logs to present logs inline
    client_replica_gc_test.go:147: -- test log scope end --
--- PASS: TestReplicaGCQueueDropReplicaDirect (5.93s)
=== RUN   TestReplicaGCQueueDropReplicaGCOnScan
    test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestReplicaGCQueueDropReplicaGCOnScan224437164
    test_log_scope.go:74: use -show-logs to present logs inline
==================
WARNING: DATA RACE
Read at 0x00c005407140 by goroutine 499:
  google.golang.org/grpc/internal/transport.(*Stream).RecvCompress()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/transport.go:338 +0xf0a
  google.golang.org/grpc.(*csAttempt).recvMsg()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/stream.go:943 +0xeca
  google.golang.org/grpc.(*clientStream).RecvMsg.func1()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/stream.go:808 +0x64
  google.golang.org/grpc.(*clientStream).withRetry()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/stream.go:666 +0xe6
  google.golang.org/grpc.(*clientStream).RecvMsg()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/stream.go:807 +0x164
  google.golang.org/grpc.invoke()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/call.go:73 +0x1a2
  github.com/cockroachdb/cockroach/pkg/util/tracing.ClientInterceptor.func2()
      /go/src/github.com/cockroachdb/cockroach/pkg/util/tracing/grpc_interceptor.go:259 +0x150
  google.golang.org/grpc.(*ClientConn).Invoke()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/call.go:35 +0x1a6
  github.com/cockroachdb/cockroach/pkg/server/serverpb.(*statusClient).ListLocalSessions()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/serverpb/status.pb.go:4757 +0x10a
  github.com/cockroachdb/cockroach/pkg/server.(*statusServer).listSessionsHelper.func2()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/status.go:2075 +0xd0
  github.com/cockroachdb/cockroach/pkg/server.(*rpcNodePaginator).queryNode()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/pagination.go:381 +0x1ea
  github.com/cockroachdb/cockroach/pkg/server.(*statusServer).paginatedIterateNodes.func2()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/status.go:2052 +0x65
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask.func2()
      /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:406 +0x189
Previous write at 0x00c005407140 by goroutine 877:
  google.golang.org/grpc/internal/transport.(*http2Client).operateHeaders()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1289 +0x2309
  google.golang.org/grpc/internal/transport.(*http2Client).reader()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1478 +0xdb1
Goroutine 499 (running) created at:
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask()
      /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:400 +0x29a
  github.com/cockroachdb/cockroach/pkg/server.(*statusServer).paginatedIterateNodes()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/status.go:2049 +0xb3b
  github.com/cockroachdb/cockroach/pkg/server.(*statusServer).listSessionsHelper()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/status.go:2101 +0x447
  github.com/cockroachdb/cockroach/pkg/server.(*apiV2Server).listSessions()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/api_v2.go:251 +0x39b
  github.com/cockroachdb/cockroach/pkg/server.(*apiV2Server).listSessions-fm()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/api_v2.go:243 +0x68
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2069 +0x51
  github.com/cockroachdb/cockroach/pkg/server.(*callCountDecorator).ServeHTTP()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/api_v2.go:193 +0x9e
  github.com/cockroachdb/cockroach/pkg/server.(*roleAuthorizationMux).ServeHTTP()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/api_v2_auth.go:436 +0x34f
  github.com/gorilla/mux.(*Router).ServeHTTP()
      /go/src/github.com/cockroachdb/cockroach/vendor/github.com/gorilla/mux/mux.go:210 +0x132
  github.com/cockroachdb/cockroach/pkg/server.(*authenticationV2Mux).ServeHTTP()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/api_v2_auth.go:333 +0x3b5
  github.com/gorilla/mux.(*Router).ServeHTTP()
      /go/src/github.com/cockroachdb/cockroach/vendor/github.com/gorilla/mux/mux.go:210 +0x132
  github.com/cockroachdb/cockroach/pkg/server.(*apiV2Server).ServeHTTP()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/api_v2.go:183 +0x6e
  net/http.(*ServeMux).ServeHTTP()
      /usr/local/go/src/net/http/server.go:2448 +0xaf
  github.com/cockroachdb/cockroach/pkg/server.(*Server).ServeHTTP()
      /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2587 +0x24d
  net/http.serverHandler.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2887 +0xca
  net/http.(*conn).serve()
      /usr/local/go/src/net/http/server.go:1952 +0x87d
Goroutine 877 (running) created at:
  google.golang.org/grpc/internal/transport.newHTTP2Client()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x1488
  google.golang.org/grpc/internal/transport.NewClientTransport()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/transport.go:580 +0x6c7
  google.golang.org/grpc.(*addrConn).createTransport()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:1323 +0x4f7
  google.golang.org/grpc.(*addrConn).tryAllAddrs()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:1253 +0x507
  google.golang.org/grpc.(*addrConn).resetTransport()
      /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:1168 +0x33b
==================
FAIL	github.com/cockroachdb/cockroach/pkg/server	272.572s

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.T-server-and-securityDB Server & Securityskipped-test

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions