cdc: changefeed TeamCity flakiness

There've been recent test failures around a couple Changefeed tests in TeamCity:
[TestChangefeedWorksOnRBRChange](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_UnitTests/3711017?), [TestChangefeedSchemaChangeAllowBackfill](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_UnitTests/3711211?buildTab=overview#%2Ftest), [TestChangefeedSchemaChangeAllowBackfill](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_UnitTests/3711439?)

On running them in roachprod, they seem to fail after around 3k runs.  The command I used to test this was:
```
make roachprod-stress CLUSTER=shiranka-stress PKG=./pkg/ccl/changefeedccl TESTS=TestChangefeedWorksOnRBRChange TESTTIMEOUT=5m TESTFLAGS="-show-logs -vmodule=helpers_test=1"
```

In investigating the RBR error it appeared to be blocking somewhere in the `eventToRow` call in `changefeed_processors.go:ConsumeEvent`

<details>
<summary>Stack trace of the error</summary>

```
initialized metamorphic constant "row-container-rows-per-chunk-shift" with value 1
initialized metamorphic constant "inverted-joiner-batch-size" with value 1
initialized metamorphic constant "zig-zag-joiner-batch-size" with value 1
initialized metamorphic constant "spilling-queue-initial-len" with value 10
initialized metamorphic constant "merge-joiner-groups-buffer" with value 14
initialized metamorphic constant "ColIndexJoin-batch-size" with value 3786002
initialized metamorphic constant "max-batch-size" with value 4440
initialized metamorphic constant "max-batch-byte-size" with value 27190907
initialized metamorphic constant "restore-worker-concurrency" with value 3
I211111 20:37:19.544408 1 (gostd) rand.go:130  [-] 1  random seed: -555974359718667819
=== RUN   TestChangefeedWorksOnRBRChange
    test_log_scope.go:76: test logs captured to: /tmp/logTestChangefeedWorksOnRBRChange001525437
=== RUN   TestChangefeedWorksOnRBRChange/kafka/format=avro
I211111 20:37:19.556851 66 server/config.go:626  [n?] 1  1 storage engine initialized
I211111 20:37:19.557393 66 server/config.go:629  [n?] 2  Pebble cache size: 128 MiB
I211111 20:37:19.557438 66 server/config.go:629  [n?] 3  store 0: in-memory, size 512 MiB
I211111 20:37:19.562958 66 1@rpc/tls.go:265  [n?] 4  server certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I211111 20:37:19.563070 66 1@rpc/tls.go:314  [n?] 5  web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
W211111 20:37:19.563247 66 server/status/runtime.go:325  [n?] 6  could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I211111 20:37:19.649784 66 1@server/server.go:1015  [n?] 7  monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I211111 20:37:19.656350 66 server/init.go:197  [n?] 8  no stores initialized
I211111 20:37:19.656445 66 server/init.go:198  [n?] 9  awaiting `cockroach init` or join with an already initialized node
I211111 20:37:19.656494 66 server/init.go:262  [n?] 10  cluster d1bb5379-944b-49d2-8e4e-f8f3d49089f2 has been created
I211111 20:37:19.656543 66 server/init.go:263  [n?] 11  allocated node ID: n1 (for self)
I211111 20:37:19.656587 66 server/init.go:264  [n?] 12  active cluster version: 21.2-10
I211111 20:37:19.656689 66 1@server/server.go:1058  [n?] 13  Sleeping till wall time 1636663039656688296 to catches up to 1636663040149779119 to ensure monotonicity. Delta: 493.090823ms
I211111 20:37:19.656758 120 1@server/server.go:1582  [n?] 14  connecting to gossip network to verify cluster ID "d1bb5379-944b-49d2-8e4e-f8f3d49089f2"
W211111 20:37:20.150269 66 1@gossip/gossip.go:1472  [n?] 15  no addresses found; use --join to specify a connected node
I211111 20:37:20.150398 66 gossip/gossip.go:401  [n1] 16  NodeDescriptor set to node_id:1 address:<network_field:"tcp" address_field:"127.0.0.1:34537" > attrs:<> locality:<tiers:<key:"region" value:"us-east-1" > > ServerVersion:<major_val:21 minor_val:2 patch:0 internal:10 > build_tag:"v22.1.0-alpha.00000000-1135-gd7e3ceb1f9" started_at:1636663040150387036 cluster_name:"" sql_address:<network_field:"tcp" address_field:"127.0.0.1:34719" >
I211111 20:37:20.157354 66 server/node.go:422  [n1] 17  initialized store s1
I211111 20:37:20.157453 66 kv/kvserver/stores.go:250  [n1] 18  read 0 node addresses from persistent storage
I211111 20:37:20.157541 66 server/node.go:499  [n1] 19  started with engine type 2
I211111 20:37:20.157576 66 server/node.go:501  [n1] 20  started with attributes []
I211111 20:37:20.157659 66 1@server/server.go:1708  [n1] 21  starting https server at 127.0.0.1:40343 (use: 127.0.0.1:40343)
I211111 20:37:20.157700 66 1@server/server.go:1713  [n1] 22  starting postgres server at 127.0.0.1:34719 (use: 127.0.0.1:34719)
I211111 20:37:20.157747 66 1@server/server.go:1715  [n1] 23  starting grpc server at 127.0.0.1:34537
I211111 20:37:20.157780 66 1@server/server.go:1716  [n1] 24  advertising CockroachDB node at 127.0.0.1:34537
I211111 20:37:20.161765 120 1@server/server.go:1585  [n1] 25  node connected via gossip
I211111 20:37:20.183663 66 1@util/log/event_log.go:32  [n1] 26 ={"Timestamp":1636663040183661249,"EventType":"node_join","NodeID":1,"StartedAt":1636663040150387036,"LastUp":1636663040150387036}
I211111 20:37:20.183754 66 sql/sqlliveness/slinstance/slinstance.go:296  [n1] 27  starting SQL liveness instance
I211111 20:37:20.191407 216 sql/sqlliveness/slstorage/slstorage.go:397  [n1] 28  inserted sqlliveness session f9c3990f0dc349c59790dd113573f0e2
I211111 20:37:20.191472 216 sql/sqlliveness/slinstance/slinstance.go:183  [n1] 29  created new SQL liveness session f9c3990f0dc349c59790dd113573f0e2
I211111 20:37:20.195199 225 sql/temporary_schema.go:512  [n1] 30  running temporary object cleanup background job
I211111 20:37:20.195647 225 sql/temporary_schema.go:556  [n1] 31  found 0 temporary schemas
I211111 20:37:20.195706 225 sql/temporary_schema.go:559  [n1] 32  early exiting temporary schema cleaner as no temporary schemas were found
I211111 20:37:20.195739 225 sql/temporary_schema.go:560  [n1] 33  completed temporary object cleanup job
I211111 20:37:20.195768 225 sql/temporary_schema.go:643  [n1] 34  temporary object cleaner next scheduled to run at 2021-11-11 21:07:20.195170888 +0000 UTC
I211111 20:37:20.195900 372 sql/sqlstats/persistedsqlstats/provider.go:137  [n1] 35  starting sql-stats-worker with initial delay: 53m34.347145047s
I211111 20:37:20.206954 404 kv/kvserver/replica_rangefeed.go:657  [n1,rangefeed=lease,s1,r6/1:/Table/{SystemCon…-11}] 36  RangeFeed closed timestamp is empty
I211111 20:37:20.250999 361 util/log/event_log.go:32  [n1,intExec=optInToDiagnosticsStatReporting] 37 ={"Timestamp":1636663040221335849,"EventType":"set_cluster_setting","Statement":"SET CLUSTER SETTING \"diagnostics.reporting.enabled\" = true","Tag":"SET CLUSTER SETTING","User":"root","ApplicationName":"$ internal-optInToDiagnosticsStatReporting","SettingName":"diagnostics.reporting.enabled","Value":"true"}
I211111 20:37:20.279568 422 migration/migrationmanager/manager.go:110  [n1,intExec=set-setting,migration-mgr] 38  no need to migrate, cluster already at newest version
I211111 20:37:20.285624 422 util/log/event_log.go:32  [n1,intExec=set-setting] 39 ={"Timestamp":1636663040270245314,"EventType":"set_cluster_setting","Statement":"SET CLUSTER SETTING version = $1","Tag":"SET CLUSTER SETTING","User":"root","ApplicationName":"$ internal-set-setting","PlaceholderValues":["'21.2-10'"],"SettingName":"version","Value":"21.2-10"}
I211111 20:37:20.315813 825 util/log/event_log.go:32  [n1,intExec=initializeClusterSecret] 40 ={"Timestamp":1636663040311873868,"EventType":"set_cluster_setting","Statement":"SET CLUSTER SETTING \"cluster.secret\" = gen_random_uuid()::STRING","Tag":"SET CLUSTER SETTING","User":"root","ApplicationName":"$ internal-initializeClusterSecret","SettingName":"cluster.secret","Value":"85170b4c-4122-4e7b-a30a-d1446fb9d85a"}
I211111 20:37:20.324300 883 5@util/log/event_log.go:32  [n1,intExec=create-default-DB] 41 ={"Timestamp":1636663040317899641,"EventType":"create_database","Statement":"CREATE DATABASE IF NOT EXISTS defaultdb","Tag":"CREATE DATABASE","User":"root","DescriptorID":50,"ApplicationName":"$ internal-create-default-DB","DatabaseName":"defaultdb"}
I211111 20:37:20.340786 889 5@util/log/event_log.go:32  [n1,intExec=create-default-DB] 42 ={"Timestamp":1636663040329024577,"EventType":"create_database","Statement":"CREATE DATABASE IF NOT EXISTS postgres","Tag":"CREATE DATABASE","User":"root","DescriptorID":51,"ApplicationName":"$ internal-create-default-DB","DatabaseName":"postgres"}
I211111 20:37:20.409224 66 server/server_sql.go:1119  [n1] 43  done ensuring all necessary startup migrations have run
I211111 20:37:20.412728 1165 jobs/job_scheduler.go:357  [n1] 44  waiting 2m0s before scheduled jobs daemon start
W211111 20:37:20.420417 110 kv/range_lookup.go:243  [n1,rangefeed=table-stats-cache,range-lookup=/Meta2/Table/20/NULL] 45  range lookup of key /Meta2/Table/20/NULL found only non-matching ranges []; retrying
I211111 20:37:20.421101 1167 server/auto_upgrade.go:55  [n1] 46  no need to upgrade, cluster already at the newest version
W211111 20:37:20.422020 109 kv/range_lookup.go:243  [n1,rangefeed=table-stats-cache,range-lookup=/Table/20] 47  range lookup of key /Table/20 found only non-matching ranges []; retrying
I211111 20:37:20.508600 10 util/admission/granter.go:860  [-] 48  CPULoad switching to period 250ms
I211111 20:37:20.531048 66 1@server/server.go:2370  [n1] 49  serving sql connections
I211111 20:37:20.604738 1198 util/log/event_log.go:32  [n1,client=127.0.0.1:60294,hostssl,user=root] 50 ={"Timestamp":1636663040585543267,"EventType":"set_cluster_setting","Statement":"SET CLUSTER SETTING \"kv.rangefeed.enabled\" = true","Tag":"SET CLUSTER SETTING","User":"root","SettingName":"kv.rangefeed.enabled","Value":"true"}
I211111 20:37:20.609859 1198 util/log/event_log.go:32  [n1,client=127.0.0.1:60294,hostssl,user=root] 51 ={"Timestamp":1636663040604948425,"EventType":"set_cluster_setting","Statement":"SET CLUSTER SETTING \"kv.closed_timestamp.target_duration\" = '1s'","Tag":"SET CLUSTER SETTING","User":"root","SettingName":"kv.closed_timestamp.target_duration","Value":"00:00:01"}
I211111 20:37:20.643219 1198 util/log/event_log.go:32  [n1,client=127.0.0.1:60294,hostssl,user=root] 52 ={"Timestamp":1636663040610064194,"EventType":"set_cluster_setting","Statement":"SET CLUSTER SETTING \"changefeed.experimental_poll_interval\" = '10ms'","Tag":"SET CLUSTER SETTING","User":"root","SettingName":"changefeed.experimental_poll_interval","Value":"00:00:00.01"}
I211111 20:37:20.650204 1198 util/log/event_log.go:32  [n1,client=127.0.0.1:60294,hostssl,user=root] 53 ={"Timestamp":1636663040643413241,"EventType":"set_cluster_setting","Statement":"SET CLUSTER SETTING \"sql.defaults.vectorize\" = \"on\"","Tag":"SET CLUSTER SETTING","User":"root","SettingName":"sql.defaults.vectorize","Value":"on"}
I211111 20:37:20.656886 1198 5@util/log/event_log.go:32  [n1,client=127.0.0.1:60294,hostssl,user=root] 54 ={"Timestamp":1636663040650415738,"EventType":"create_database","Statement":"CREATE DATABASE d","Tag":"CREATE DATABASE","User":"root","DescriptorID":52,"DatabaseName":"d"}
I211111 20:37:20.690392 1198 sql/table.go:98  [n1,client=127.0.0.1:60294,hostssl,user=root] 55  queued new database schema change job 709730764979339265 for database 52
I211111 20:37:20.745656 1198 5@util/log/event_log.go:32  [n1,client=127.0.0.1:60294,hostssl,user=root] 56 ={"Timestamp":1636663040659028063,"EventType":"create_type","Statement":"ALTER DATABASE d PRIMARY REGION \"us-east-1\"","Tag":"ALTER DATABASE PRIMARY REGION","User":"root","DescriptorID":53,"TypeName":"d.public.crdb_internal_region"}
I211111 20:37:20.745770 1198 5@util/log/event_log.go:32  [n1,client=127.0.0.1:60294,hostssl,user=root] 57 ={"Timestamp":1636663040659028063,"EventType":"alter_database_primary_region","Statement":"ALTER DATABASE d PRIMARY REGION \"us-east-1\"","Tag":"ALTER DATABASE PRIMARY REGION","User":"root","DescriptorID":52,"DatabaseName":"d","PrimaryRegionName":"\"us-east-1\""}
I211111 20:37:20.746057 1198 jobs/registry.go:349  [n1,client=127.0.0.1:60294,hostssl,user=root] 58  scheduled jobs [709730764979339265]
I211111 20:37:20.757663 1739 jobs/adopt.go:247  [n1] 59  job 709730764979339265: resuming execution
I211111 20:37:20.765721 1742 jobs/registry.go:1172  [n1] 60  SCHEMA CHANGE job 709730764979339265: stepping through state running with error: <nil>
I211111 20:37:20.778650 1742 sql/schema_changer.go:582  [n1,job=709730764979339265,scExec,id=52] 61  schema change on "d" (v2) starting execution...
I211111 20:37:20.778745 1742 sql/schema_changer.go:1021  [n1,job=709730764979339265,scExec,id=52] 62  waiting for a single version...
I211111 20:37:20.781752 1742 sql/schema_changer.go:1027  [n1,job=709730764979339265,scExec,id=52] 63  waiting for a single version... done (at v 2)
I211111 20:37:20.781843 1742 jobs/registry.go:1172  [n1] 64  SCHEMA CHANGE job 709730764979339265: stepping through state succeeded with error: <nil>
I211111 20:37:20.837762 1198 util/log/event_log.go:32  [n1,client=127.0.0.1:60294,hostssl,user=root] 65 ={"Timestamp":1636663040823752446,"EventType":"set_cluster_setting","Statement":"SET CLUSTER SETTING \"kv.closed_timestamp.target_duration\" = '50ms'","Tag":"SET CLUSTER SETTING","User":"root","SettingName":"kv.closed_timestamp.target_duration","Value":"00:00:00.05"}
=== RUN   TestChangefeedWorksOnRBRChange/kafka/format=avro/regional_by_row_as_change_works
I211111 20:37:20.845207 1198 sql/type_change.go:147  [n1,client=127.0.0.1:60294,hostssl,user=root] 66  queued new type change job 709730765521551361 for type 53
I211111 20:37:20.857658 1198 sql/type_change.go:147  [n1,client=127.0.0.1:60294,hostssl,user=root] 67  queued new type change job 709730765562347521 for type 54
I211111 20:37:20.891908 1198 5@util/log/event_log.go:32  [n1,client=127.0.0.1:60294,hostssl,user=root] 68 ={"Timestamp":1636663040838662345,"EventType":"create_table","Statement":"CREATE TABLE d.public.rbr (a INT8 PRIMARY KEY, b INT8, region crdb_internal_region NOT NULL DEFAULT 'us-east-1')","Tag":"CREATE TABLE","User":"root","DescriptorID":55,"TableName":"d.public.rbr"}
I211111 20:37:20.896941 1198 jobs/registry.go:349  [n1,client=127.0.0.1:60294,hostssl,user=root] 69  scheduled jobs [709730765521551361 709730765562347521]
I211111 20:37:20.902205 2050 kv/kvserver/replica_command.go:395  [n1,split,s1,r43/1:/{Table/47-Max}] 70  initiating a split of this range at key /Table/55 [r44] (span config)
I211111 20:37:20.926624 1973 jobs/adopt.go:247  [n1] 71  job 709730765562347521: resuming execution
I211111 20:37:20.926669 1972 jobs/adopt.go:247  [n1] 72  job 709730765521551361: resuming execution
I211111 20:37:20.929332 1975 jobs/registry.go:1172  [n1] 73  TYPEDESC SCHEMA CHANGE job 709730765562347521: stepping through state running with error: <nil>
I211111 20:37:20.942215 1872 jobs/registry.go:1172  [n1] 74  TYPEDESC SCHEMA CHANGE job 709730765521551361: stepping through state running with error: <nil>
I211111 20:37:20.951364 1975 sql/schema_changer.go:1021  [n1,job=709730765562347521] 75  waiting for a single version...
I211111 20:37:20.956298 1975 sql/schema_changer.go:1027  [n1,job=709730765562347521] 76  waiting for a single version... done (at v 2)
I211111 20:37:20.956382 1975 jobs/registry.go:1172  [n1] 77  TYPEDESC SCHEMA CHANGE job 709730765562347521: stepping through state succeeded with error: <nil>
I211111 20:37:20.965268 1872 sql/schema_changer.go:1021  [n1,job=709730765521551361] 78  waiting for a single version...
I211111 20:37:20.967878 1872 sql/schema_changer.go:1027  [n1,job=709730765521551361] 79  waiting for a single version... done (at v 2)
I211111 20:37:20.967936 1872 sql/schema_changer.go:1021  [n1,job=709730765521551361] 80  waiting for a single version...
I211111 20:37:20.970486 1872 sql/schema_changer.go:1027  [n1,job=709730765521551361] 81  waiting for a single version... done (at v 2)
I211111 20:37:20.970546 1872 jobs/registry.go:1172  [n1] 82  TYPEDESC SCHEMA CHANGE job 709730765521551361: stepping through state succeeded with error: <nil>
I211111 20:37:21.141866 2218 jobs/registry.go:1172  [n1] 83  CHANGEFEED job 709730766246510593: stepping through state running with error: <nil>
I211111 20:37:21.162494 1838 ccl/changefeedccl/helpers_test.go:77  [-] 84  msg rbr: {"a":{"long":0}}->{"after":{"rbr":{"a":{"long":0},"b":null,"region":{"string":"us-east-1"}}}} ()
I211111 20:37:21.205745 1838 ccl/changefeedccl/helpers_test.go:77  [-] 85  msg rbr: {"a":{"long":1}}->{"after":{"rbr":{"a":{"long":1},"b":{"long":2},"region":{"string":"us-east-1"}}}} ()
I211111 20:37:21.248596 1198 sql/table.go:165  [n1,client=127.0.0.1:60294,hostssl,user=root] 86  queued new schema-change job 709730766843379713 for table 55, mutation 1
I211111 20:37:21.273485 1198 5@util/log/event_log.go:32  [n1,client=127.0.0.1:60294,hostssl,user=root] 87 ={"Timestamp":1636663041222365255,"EventType":"alter_table","Statement":"ALTER TABLE d.public.rbr SET LOCALITY REGIONAL BY ROW AS region","Tag":"ALTER TABLE SET LOCALITY","User":"root","DescriptorID":55,"TableName":"rbr"}
I211111 20:37:21.273679 1198 jobs/registry.go:349  [n1,client=127.0.0.1:60294,hostssl,user=root] 88  scheduled jobs [709730766843379713]
I211111 20:37:21.279753 2303 sql/catalog/lease/descriptor_state.go:139  [n1] 89  new lease: tabledesc.immutable: {ID: 55, Version: 2, ModificationTime: "1636663041.247337663,1", ParentID: 52, ParentSchemaID: 29, State: PUBLIC, NextColumnID: 4, Columns: [{ID: 1, TypeID: 20, Null: false}, {ID: 2, TypeID: 20, Null: true}, {ID: 3, TypeID: 100053, Null: false, HasDefault: true}], NextFamilyID: 1, Families: [{ID: 0, Columns: [1, 2, 3]}], MutationJobs: [{MutationID: 1, JobID: 709730766843379713}], Mutations: [{MutationID: 1, Direction: ADD, State: DELETE_ONLY, Index: {ID: 2, Unique: true, KeyColumns: [{ID: 3, Dir: ASC}, {ID: 1, Dir: ASC}], StoreColumns: [2], State: ADD, MutationID: 1}}, {MutationID: 1, Direction: ADD, State: DELETE_ONLY, PrimaryKeySwap: {OldPrimaryIndexID: 1, OldIndexes: [], NewPrimaryIndexID: 2, NewIndexes: []}}], PrimaryIndex: 1, NextIndexID: 3, Indexes: [{ID: 1, Unique: true, KeyColumns: [{ID: 1, Dir: ASC}], StoreColumns: [2, 3]}]}
I211111 20:37:21.287232 2328 jobs/adopt.go:247  [n1] 90  job 709730766843379713: resuming execution
I211111 20:37:21.290056 2330 jobs/registry.go:1172  [n1] 91  SCHEMA CHANGE job 709730766843379713: stepping through state running with error: <nil>
I211111 20:37:21.295109 2330 sql/schema_changer.go:582  [n1,job=709730766843379713,scExec,id=55,mutation=1] 92  schema change on "rbr" (v2) starting execution...
I211111 20:37:21.307570 2330 sql/schema_changer.go:897  [n1,job=709730766843379713,scExec,id=55,mutation=1] 93  stepping through state machine
I211111 20:37:21.338017 2452 sql/catalog/lease/descriptor_state.go:139  [n1,job=709730766246510593] 94  new lease: tabledesc.immutable: {ID: 55, Version: 3, ModificationTime: "1636663041.317952343,1", ParentID: 52, ParentSchemaID: 29, State: PUBLIC, NextColumnID: 4, Columns: [{ID: 1, TypeID: 20, Null: false}, {ID: 2, TypeID: 20, Null: true}, {ID: 3, TypeID: 100053, Null: false, HasDefault: true}], NextFamilyID: 1, Families: [{ID: 0, Columns: [1, 2, 3]}], MutationJobs: [{MutationID: 1, JobID: 709730766843379713}], Mutations: [{MutationID: 1, Direction: ADD, State: DELETE_AND_WRITE_ONLY, Index: {ID: 2, Unique: true, KeyColumns: [{ID: 3, Dir: ASC}, {ID: 1, Dir: ASC}], StoreColumns: [2], State: ADD, MutationID: 1}}, {MutationID: 1, Direction: ADD, State: DELETE_AND_WRITE_ONLY, PrimaryKeySwap: {OldPrimaryIndexID: 1, OldIndexes: [], NewPrimaryIndexID: 2, NewIndexes: []}}], PrimaryIndex: 1, NextIndexID: 3, Indexes: [{ID: 1, Unique: true, KeyColumns: [{ID: 1, Dir: ASC}], StoreColumns: [2, 3]}]}
I211111 20:37:21.342006 2330 sql/catalog/lease/lease.go:141  [n1,job=709730766843379713,scExec,id=55,mutation=1] 95  waiting for 1 leases to expire: desc=[{rbr 55 2}]
I211111 20:37:21.356780 2336 kv/kvserver/replica_command.go:395  [n1,split,s1,r44/1:/{Table/55-Max}] 96  initiating a split of this range at key /Table/55/2/"\x80" [r45] (span config)
I211111 20:37:21.362810 2598 kv/kvserver/replica_command.go:395  [n1,split,s1,r45/1:/{Table/55/2/"…-Max}] 97  initiating a split of this range at key /Table/55/2/"\x80"/PrefixEnd [r46] (span config)
I211111 20:37:21.421898 2330 sql/schema_changer.go:982  [n1,job=709730766843379713,scExec,id=55,mutation=1] 98  finished stepping through state machine
I211111 20:37:21.430693 2330 sql/backfill.go:249  [n1,job=709730766843379713,scExec,id=55,mutation=1] 99  running backfill for "rbr", v=3
I211111 20:37:21.430801 2330 sql/backfill.go:1842  [n1,job=709730766843379713,scExec,id=55,mutation=1] 100  backfilling 1 indexes
I211111 20:37:21.430923 2330 kv/kvserver/replica_command.go:395  [n1,s1,r44/1:/Table/55{-/2/"\x80"}] 101  initiating a split of this range at key /Table/55/2 [r47] (manual)
I211111 20:37:21.451530 2330 sql/backfill.go:925  [n1,job=709730766843379713,scExec,id=55,mutation=1] 102  starting scan of target index as of 1636663041.451529827,0...
I211111 20:37:21.451931 2330 sql/backfill.go:950  [n1,job=709730766843379713,scExec,id=55,mutation=1] 103  persisting target safe write time 1636663041.451529827,0...
I211111 20:37:21.470094 2330 sql/backfill.go:1866  [n1,job=709730766843379713,scExec,id=55,mutation=1] 104  finished backfilling indexes
I211111 20:37:21.470215 2330 sql/backfill.go:1371  [n1,job=709730766843379713,scExec,id=55,mutation=1] 105  validating new indexes
I211111 20:37:21.479398 2706 sql/backfill.go:1719  [n1,job=709730766843379713,scExec,id=55,mutation=1] 106  validation: index rbr/new_primary_key row count = 2, time so far 2.385379ms
I211111 20:37:21.497310 2707 sql/backfill.go:1812  [n1,job=709730766843379713,scExec,id=55,mutation=1] 107  validation: table rbr row count = 2, took 20.938582ms
I211111 20:37:21.497434 2330 sql/backfill.go:1434  [n1,job=709730766843379713,scExec,id=55,mutation=1] 108  finished validating new indexes
I211111 20:37:21.497489 2330 sql/backfill.go:369  [n1,job=709730766843379713,scExec,id=55,mutation=1] 109  completed backfill for "rbr", v=3
I211111 20:37:21.501199 2330 sql/schema_changer.go:1427  [n1,job=709730766843379713,scExec,id=55,mutation=1] 110  marking schema change as complete
I211111 20:37:21.512157 2330 sql/schema_changer.go:2358  [n1,job=709730766843379713,scExec,id=55,mutation=1] 111  created job 709730767702851585 to drop previous columns and indexes
I211111 20:37:21.520227 2330 5@util/log/event_log.go:32  [n1,job=709730766843379713,scExec,id=55,mutation=1] 112 ={"Timestamp":1636663041501309989,"EventType":"finish_schema_change","InstanceID":1,"DescriptorID":55,"MutationID":1}
I211111 20:37:21.523649 2669 sql/catalog/lease/descriptor_state.go:139  [n1,job=709730766246510593] 113  new lease: typedesc.immutable: {ID: 53, Version: 3, ModificationTime: "1636663041.505611019,1", ParentID: 52, ParentSchemaID: 29, State: PUBLIC, Kind: MULTIREGION_ENUM, NumEnumMembers: 1, ArrayTypeID: 54, ReferencingDescriptorIDs: [55]}
I211111 20:37:21.571953 2734 jobs/adopt.go:247  [n1] 114  job 709730767702851585: resuming execution
I211111 20:37:21.576206 2736 jobs/registry.go:1172  [n1] 115  SCHEMA CHANGE job 709730767702851585: stepping through state running with error: <nil>
I211111 20:37:21.587437 2822 sql/catalog/lease/descriptor_state.go:139  [n1,job=709730766246510593] 116  new lease: tabledesc.immutable: {ID: 55, Version: 4, ModificationTime: "1636663041.505611019,1", ParentID: 52, ParentSchemaID: 29, State: PUBLIC, NextColumnID: 4, Columns: [{ID: 1, TypeID: 20, Null: false}, {ID: 2, TypeID: 20, Null: true}, {ID: 3, TypeID: 100053, Null: false, HasDefault: true}], NextFamilyID: 1, Families: [{ID: 0, Columns: [1, 2, 3]}], MutationJobs: [{MutationID: 2, JobID: 709730767702851585}], Mutations: [{MutationID: 2, Direction: DROP, State: DELETE_AND_WRITE_ONLY, Index: {ID: 1, Unique: true, KeyColumns: [{ID: 1, Dir: ASC}], StoreColumns: [2, 3], State: DROP, MutationID: 2}}], PrimaryIndex: 2, NextIndexID: 3, Indexes: [{ID: 2, Unique: true, KeyColumns: [{ID: 3, Dir: ASC}, {ID: 1, Dir: ASC}], StoreColumns: [2]}]}
I211111 20:37:21.637480 2330 sql/catalog/lease/lease.go:141  [n1,job=709730766843379713,scExec,id=55,mutation=1] 117  waiting for 1 leases to expire: desc=[{crdb_internal_region 53 2}]
I211111 20:37:21.651155 2736 sql/schema_changer.go:582  [n1,job=709730767702851585,scExec,id=55,mutation=2] 118  schema change on "rbr" (v4) starting execution...
I211111 20:37:21.676315 2736 sql/schema_changer.go:897  [n1,job=709730767702851585,scExec,id=55,mutation=2] 119  stepping through state machine
I211111 20:37:21.746508 2870 sql/catalog/lease/descriptor_state.go:139  [n1] 120  new lease: tabledesc.immutable: {ID: 55, Version: 5, ModificationTime: "1636663041.692520499,1", ParentID: 52, ParentSchemaID: 29, State: PUBLIC, NextColumnID: 4, Columns: [{ID: 1, TypeID: 20, Null: false}, {ID: 2, TypeID: 20, Null: true}, {ID: 3, TypeID: 100053, Null: false, HasDefault: true}], NextFamilyID: 1, Families: [{ID: 0, Columns: [1, 2, 3]}], MutationJobs: [{MutationID: 2, JobID: 709730767702851585}], Mutations: [{MutationID: 2, Direction: DROP, State: DELETE_ONLY, Index: {ID: 1, Unique: true, KeyColumns: [{ID: 1, Dir: ASC}], StoreColumns: [2, 3], State: DROP, MutationID: 2}}], PrimaryIndex: 2, NextIndexID: 3, Indexes: [{ID: 2, Unique: true, KeyColumns: [{ID: 3, Dir: ASC}, {ID: 1, Dir: ASC}], StoreColumns: [2]}]}
I211111 20:37:21.749928 2330 sql/catalog/lease/lease.go:141  [n1,job=709730766843379713,scExec,id=55,mutation=1] 121  waiting for 1 leases to expire: desc=[{rbr 55 4}]
I211111 20:37:21.754658 2736 sql/catalog/lease/lease.go:141  [n1,job=709730767702851585,scExec,id=55,mutation=2] 122  waiting for 1 leases to expire: desc=[{rbr 55 4}]
I211111 20:37:21.771154 2188 ccl/changefeedccl/kvfeed/kv_feed.go:120  [n1,job=709730766246510593] 123  stopping kv feed due to schema change at 1636663041.505611019,1
I211111 20:37:21.797205 2330 sql/schema_changer.go:1021  [n1,job=709730766843379713,scExec,id=55,mutation=1] 124  waiting for a single version...
I211111 20:37:21.800363 2330 sql/schema_changer.go:1027  [n1,job=709730766843379713,scExec,id=55,mutation=1] 125  waiting for a single version... done (at v 5)
I211111 20:37:21.800450 2330 jobs/registry.go:1172  [n1] 126  SCHEMA CHANGE job 709730766843379713: stepping through state succeeded with error: <nil>
W211111 20:37:21.802021 2218 ccl/changefeedccl/changefeed_stmt.go:745  [n1,job=709730766246510593] 127  WARNING: CHANGEFEED job 709730766246510593 encountered retryable error: retryable changefeed error: schema change occurred at 1636663041505611019.0000000001
I211111 20:37:21.838794 2736 sql/schema_changer.go:982  [n1,job=709730767702851585,scExec,id=55,mutation=2] 128  finished stepping through state machine
I211111 20:37:21.842046 2736 sql/backfill.go:249  [n1,job=709730767702851585,scExec,id=55,mutation=2] 129  running backfill for "rbr", v=5
I211111 20:37:21.842161 2736 sql/backfill.go:369  [n1,job=709730767702851585,scExec,id=55,mutation=2] 130  completed backfill for "rbr", v=5
I211111 20:37:21.842203 2736 sql/schema_changer.go:1427  [n1,job=709730767702851585,scExec,id=55,mutation=2] 131  marking schema change as complete
I211111 20:37:21.845431 2736 sql/schema_changer.go:1005  [n1,job=709730767702851585,scExec,id=55,mutation=2] 132  created index GC job 709730768794583041
I211111 20:37:21.856646 2736 5@util/log/event_log.go:32  [n1,job=709730767702851585,scExec,id=55,mutation=2] 133 ={"Timestamp":1636663041842240606,"EventType":"finish_schema_change","InstanceID":1,"DescriptorID":55,"MutationID":2}
I211111 20:37:21.858830 3021 sql/catalog/lease/descriptor_state.go:139  [n1] 134  new lease: typedesc.immutable: {ID: 53, Version: 4, ModificationTime: "1636663041.842240606,0", ParentID: 52, ParentSchemaID: 29, State: PUBLIC, Kind: MULTIREGION_ENUM, NumEnumMembers: 1, ArrayTypeID: 54, ReferencingDescriptorIDs: [55]}
I211111 20:37:21.870689 2736 sql/catalog/lease/lease.go:141  [n1,job=709730767702851585,scExec,id=55,mutation=2] 135  waiting for 1 leases to expire: desc=[{crdb_internal_region 53 3}]
I211111 20:37:21.874256 3076 sql/catalog/lease/descriptor_state.go:139  [n1] 136  new lease: tabledesc.immutable: {ID: 55, Version: 6, ModificationTime: "1636663041.842240606,0", ParentID: 52, ParentSchemaID: 29, State: PUBLIC, NextColumnID: 4, Columns: [{ID: 1, TypeID: 20, Null: false}, {ID: 2, TypeID: 20, Null: true}, {ID: 3, TypeID: 100053, Null: false, HasDefault: true}], NextFamilyID: 1, Families: [{ID: 0, Columns: [1, 2, 3]}], PrimaryIndex: 2, NextIndexID: 3, Indexes: [{ID: 2, Unique: true, KeyColumns: [{ID: 3, Dir: ASC}, {ID: 1, Dir: ASC}], StoreColumns: [2]}]}
I211111 20:37:21.904760 3169 jobs/adopt.go:247  [n1] 137  job 709730768794583041: resuming execution
I211111 20:37:21.907279 3203 jobs/registry.go:1172  [n1] 138  SCHEMA CHANGE GC job 709730768794583041: stepping through state running with error: <nil>
I211111 20:37:21.931145 3203 sql/gcjob/refresh_statuses.go:213  [n1,job=709730768794583041] 139  a timestamp protection delayed GC of index 1 from table 55
I211111 20:37:21.996875 55 1@gossip/gossip.go:1488  [n1] 140  node has connected to cluster via gossip
I211111 20:37:21.997061 55 kv/kvserver/stores.go:269  [n1] 141  wrote 0 node addresses to persistent storage
I211111 20:37:21.998271 2736 sql/schema_changer.go:1021  [n1,job=709730767702851585,scExec,id=55,mutation=2] 142  waiting for a single version...
I211111 20:37:22.022582 2736 sql/schema_changer.go:1027  [n1,job=709730767702851585,scExec,id=55,mutation=2] 143  waiting for a single version... done (at v 6)
I211111 20:37:22.022682 2736 jobs/registry.go:1172  [n1] 144  SCHEMA CHANGE job 709730767702851585: stepping through state succeeded with error: <nil>
I211111 20:37:22.054385 3272 jobs/adopt.go:247  [n1] 145  job 709730767702851585: resuming execution
I211111 20:37:37.554697 181 kv/kvserver/replica_rangefeed.go:659  [n1,s1,r45/1:/Table/55/2/"\x80"{-/Pr…}] 146  RangeFeed closed timestamp 1636663057.302999944,0 is behind by 251.690825ms
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147  
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +__level_____count____size___score______in__ingest(sz_cnt)____move(sz_cnt)___write(sz_cnt)____read___r-amp___w-amp
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +    WAL         3   1.3 M       -   1.2 M       -       -       -       -   1.3 M       -       -       -     1.0
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +      0         0     0 B    0.00     0 B     0 B       0     0 B       0     0 B       0     0 B       0     0.0
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +      1         0     0 B    0.00     0 B     0 B       0     0 B       0     0 B       0     0 B       0     0.0
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +      2         0     0 B    0.00     0 B     0 B       0     0 B       0     0 B       0     0 B       0     0.0
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +      3         0     0 B    0.00     0 B     0 B       0     0 B       0     0 B       0     0 B       0     0.0
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +      4         0     0 B    0.00     0 B     0 B       0     0 B       0     0 B       0     0 B       0     0.0
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +      5         0     0 B    0.00     0 B     0 B       0     0 B       0     0 B       0     0 B       0     0.0
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +      6         0     0 B       -     0 B     0 B       0     0 B       0     0 B       0     0 B       0     0.0
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +  total         0     0 B       -   1.3 M     0 B       0     0 B       0   1.3 M       0     0 B       0     1.0
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +  flush         0
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +compact         0     0 B             0 B          (size == estimated-debt, in = in-progress-bytes)
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +  ctype         0       0       0       0       0  (default, delete, elision, move, read)
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 + memtbl         3   1.8 M
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +zmemtbl         0     0 B
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 +   ztbl         0     0 B
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 + bcache         0     0 B    0.0%  (score == hit-rate)
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 + tcache         0     0 B    0.0%  (score == hit-rate)
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 + titers         0
I211111 20:37:40.168583 183 kv/kvserver/store.go:2833  [n1,s1] 147 + filter         -       -    0.0%  (score == utility)
I211111 20:37:43.223664 181 kv/kvserver/replica_rangefeed.go:659  [n1,s1,r46/1:/{Table/55/2/"…-Max}] 148  RangeFeed closed timestamp 1636663062.971813056,0 is behind by 251.843755ms
I211111 20:37:49.338575 9373 kv/kvserver/replica_consistency.go:254  [n1,consistencyChecker,s1,r4/1:/System{/tsd-tse}] 149  triggering stats recomputation to resolve delta of {ContainsEstimates:7024 LastUpdateNanos:1636663060564744700 IntentAge:0 GCBytesAge:0 LiveBytes:-21968 LiveCount:-1173 KeyBytes:-61675 KeyCount:-1173 ValBytes:39707 ValCount:-1173 IntentBytes:0 IntentCount:0 SeparatedIntentCount:0 SysBytes:0 SysCount:0 AbortSpanBytes:0}
I211111 20:37:49.535054 181 kv/kvserver/replica_rangefeed.go:659  [n1,s1,r42/1:/Table/4{6-7}] 150  RangeFeed closed timestamp 1636663069.284590565,0 is behind by 250.456491ms
I211111 20:37:54.592846 181 kv/kvserver/replica_rangefeed.go:659  [n1,s1,r45/1:/Table/55/2/"\x80"{-/Pr…}] 151  RangeFeed closed timestamp 1636663074.338704368,0 is behind by 254.131289ms
I211111 20:38:20.281516 16270 jobs/registry.go:1172  [n1] 152  AUTO CREATE STATS job 709730960186769409: stepping through state running with error: <nil>
I211111 20:38:20.359364 16270 jobs/registry.go:1172  [n1] 153  AUTO CREATE STATS job 709730960186769409: stepping through state succeeded with error: <nil>
I211111 20:38:32.466495 19032 kv/kvserver/replica_consistency.go:254  [n1,consistencyChecker,s1,r45/1:/Table/55/2/"\x80"{-/Pr…}] 154  triggering stats recomputation to resolve delta of {ContainsEstimates:2 LastUpdateNanos:1636663042391045600 IntentAge:0 GCBytesAge:0 LiveBytes:0 LiveCount:0 KeyBytes:0 KeyCount:0 ValBytes:0 ValCount:0 IntentBytes:0 IntentCount:0 SeparatedIntentCount:0 SysBytes:0 SysCount:0 AbortSpanBytes:0}
I211111 20:38:50.573704 181 kv/kvserver/replica_rangefeed.go:659  [n1,s1,r42/1:/Table/4{6-7}] 155  RangeFeed closed timestamp 1636663130.311961341,0 is behind by 261.736169ms
I211111 20:39:03.606658 181 kv/kvserver/replica_rangefeed.go:659  [n1,s1,r47/1:/Table/55/2{-/"\x80"}] 156  RangeFeed closed timestamp 1636663143.318717073,0 is behind by 287.935174ms
I211111 20:42:05.250649 181 kv/kvserver/replica_rangefeed.go:659  [n1,s1,r45/1:/Table/55/2/"\x80"{-/Pr…}] 157  RangeFeed closed timestamp 1636663324.999722173,0 is behind by 250.920097ms
panic: test timed out after 5m0s

goroutine 69523 [running]:
testing.(*M).startAlarm.func1()
	/usr/local/go/src/testing/testing.go:1700 +0xe5
created by time.goFunc
	/usr/local/go/src/time/sleep.go:180 +0x45

goroutine 1 [chan receive, 5 minutes]:
testing.(*T).Run(0xc000103200, 0x51205bc, 0x1e, 0x5770eb8, 0x4c3f01)
	/usr/local/go/src/testing/testing.go:1239 +0x2da
testing.runTests.func1(0xc000102a80)
	/usr/local/go/src/testing/testing.go:1511 +0x78
testing.tRunner(0xc000102a80, 0xc000de9d60)
	/usr/local/go/src/testing/testing.go:1193 +0xef
testing.runTests(0xc0007e4690, 0x8e136e0, 0x63, 0x63, 0xc05b7deae076639f, 0x45e0d45833, 0x91a1b40, 0xc05b7d9fe072d94d)
	/usr/local/go/src/testing/testing.go:1509 +0x2fe
testing.(*M).Run(0xc000c00480, 0x0)
	/usr/local/go/src/testing/testing.go:1417 +0x1eb
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.TestMain(0xc000c00480)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/main_test.go:31 +0xde
main.main()
	_testmain.go:293 +0x165

goroutine 5 [chan receive]:
github.com/cockroachdb/cockroach/pkg/util/log.flushDaemon()
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:75 +0x74
created by github.com/cockroachdb/cockroach/pkg/util/log.init.5
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:41 +0x35

goroutine 6 [chan receive, 5 minutes]:
github.com/cockroachdb/cockroach/pkg/util/log.signalFlusher()
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:98 +0x12c
created by github.com/cockroachdb/cockroach/pkg/util/log.init.5
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:42 +0x4d

goroutine 20 [syscall, 5 minutes]:
os/signal.signal_recv(0x0)
	/usr/local/go/src/runtime/sigqueue.go:168 +0xa5
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:23 +0x25
created by os/signal.Notify.func1.1
	/usr/local/go/src/os/signal/signal.go:151 +0x45

goroutine 51 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc000c00000)
	/go/src/github.com/cockroachdb/cockroach/vendor/go.opencensus.io/stats/view/worker.go:276 +0xcd
created by go.opencensus.io/stats/view.init.0
	/go/src/github.com/cockroachdb/cockroach/vendor/go.opencensus.io/stats/view/worker.go:34 +0x68

goroutine 39 [chan receive, 5 minutes]:
github.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc00018c4e0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149
created by github.com/klauspost/compress/zstd.newBlockDec
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173

goroutine 10 [chan receive]:
github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0.func1()
	/go/src/github.com/cockroachdb/cockroach/pkg/util/goschedstats/runnable.go:165 +0x16b
created by github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0
	/go/src/github.com/cockroachdb/cockroach/pkg/util/goschedstats/runnable.go:157 +0x35

goroutine 38 [chan receive, 5 minutes]:
github.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc00018c410)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149
created by github.com/klauspost/compress/zstd.newBlockDec
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173

goroutine 14 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/util/log.(*fileSink).gcDaemon(0xc000828180, 0x688b1e8, 0xc000c30ac0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/file_log_gc.go:25 +0x8e
created by github.com/cockroachdb/cockroach/pkg/util/log.ApplyConfig
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/flags.go:297 +0xe2f

goroutine 40 [chan receive, 5 minutes]:
github.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc00018c820)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149
created by github.com/klauspost/compress/zstd.newBlockDec
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173

goroutine 41 [chan receive, 5 minutes]:
github.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc00018c8f0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149
created by github.com/klauspost/compress/zstd.newBlockDec
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173

goroutine 52 [chan receive, 4 minutes]:
testing.(*T).Run(0xc000103800, 0x50e73d5, 0x11, 0xc000c30b40, 0xc000c30b40)
	/usr/local/go/src/testing/testing.go:1239 +0x2da
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.TestChangefeedWorksOnRBRChange(0xc000103200)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_test.go:1931 +0x195
testing.tRunner(0xc000103200, 0x5770eb8)
	/usr/local/go/src/testing/testing.go:1193 +0xef
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1238 +0x2b3

goroutine 15 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/util/log.(*fileSink).gcDaemon(0xc000828240, 0x688b1e8, 0xc000c30ac0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/file_log_gc.go:25 +0x8e
created by github.com/cockroachdb/cockroach/pkg/util/log.ApplyConfig
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/flags.go:297 +0xe2f

goroutine 16 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/util/log.(*fileSink).gcDaemon(0xc000828540, 0x688b1e8, 0xc000c30ac0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/file_log_gc.go:25 +0x8e
created by github.com/cockroachdb/cockroach/pkg/util/log.ApplyConfig
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/flags.go:297 +0xe2f

goroutine 66 [chan receive, 4 minutes]:
testing.(*T).Run(0xc00249f800, 0x5128b5d, 0x1f, 0xc0025acce0, 0x1)
	/usr/local/go/src/testing/testing.go:1239 +0x2da
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.TestChangefeedWorksOnRBRChange.func2(0xc000103800, 0xc000f5a340, 0x683f948, 0xc002697a40)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_test.go:1895 +0x19c
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.kafkaTestWithOptions.func1(0xc000103800)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/helpers_test.go:589 +0x1c5
testing.tRunner(0xc000103800, 0xc000c30b40)
	/usr/local/go/src/testing/testing.go:1193 +0xef
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1238 +0x2b3

goroutine 67 [chan receive, 4 minutes]:
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x688b290, 0xc000b187e0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:293 +0x76
runtime/pprof.Do(0x688b220, 0xc000b187e0, 0xc00007e5a0, 0x1, 0x1, 0xc000612fb8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcf
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc0008ba500)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:292 +0x89
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:284 +0xa8

goroutine 68 [chan receive, 4 minutes]:
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x688b290, 0xc000b186f0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:293 +0x76
runtime/pprof.Do(0x688b220, 0xc000b186f0, 0xc00007e5a0, 0x1, 0x1, 0xc0006137b8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcf
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc0008ba640)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:292 +0x89
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:284 +0xa8

goroutine 69 [chan receive, 4 minutes]:
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x688b290, 0xc000b188a0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:293 +0x76
runtime/pprof.Do(0x688b220, 0xc000b188a0, 0xc00007e5a0, 0x1, 0x1, 0xc000613fb8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcf
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc0008ba780)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:292 +0x89
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:284 +0xa8

goroutine 70 [chan receive, 4 minutes]:
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x688b290, 0xc000ad2270)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:293 +0x76
runtime/pprof.Do(0x688b220, 0xc000ad2270, 0xc00007e5a0, 0x1, 0x1, 0xc0000907b8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcf
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc0008ba8c0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:292 +0x89
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:284 +0xa8

goroutine 36183 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc000ea80e0, 0x7f32000001df)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc000ea80d0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/pebble/record.(*flusherCond).Wait(...)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/record/log_writer.go:193
github.com/cockroachdb/pebble/record.(*LogWriter).flushLoop(0xc000ea8000, 0x688b290, 0xc0021ad230)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/record/log_writer.go:396 +0x10b
runtime/pprof.Do(0x688b220, 0xc0021ad230, 0xc00007e4a0, 0x1, 0x1, 0xc0006ee7b8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcf
github.com/cockroachdb/pebble/record.NewLogWriter.func2(0xc000ea8000)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/record/log_writer.go:314 +0x89
created by github.com/cockroachdb/pebble/record.NewLogWriter
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/record/log_writer.go:313 +0x2ea

goroutine 21 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2a50)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2a50, 0xc000c30b80, 0x0, 0xc0004dad50)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 73 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/rpc.NewContext.func1(0x688b290, 0xc000ae15c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:416 +0x51
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc000ae15c0, 0xc000c30b80, 0x0, 0xc0007e5800)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 74 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/security.(*CertificateManager).RegisterSignalHandler.func1(0xc000a5c5a0, 0x688b220, 0xc000122010, 0xc00096dc20)
	/go/src/github.com/cockroachdb/cockroach/pkg/security/certificate_manager.go:241 +0x127
created by github.com/cockroachdb/cockroach/pkg/security.(*CertificateManager).RegisterSignalHandler
	/go/src/github.com/cockroachdb/cockroach/pkg/security/certificate_manager.go:238 +0x65

goroutine 75 [select]:
github.com/cockroachdb/cockroach/pkg/gossip.newInfoStore.func1(0x688b290, 0xc000c04750)
	/go/src/github.com/cockroachdb/cockroach/pkg/gossip/infostore.go:190 +0x14d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc000c04750, 0xc000c30b80, 0x0, 0xc000acd520)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 76 [select]:
github.com/cockroachdb/cockroach/pkg/util/admission.makeWorkQueue.func1(0xc0008bb680, 0xc00010eb40)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/work_queue.go:217 +0x9c
created by github.com/cockroachdb/cockroach/pkg/util/admission.makeWorkQueue
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/work_queue.go:213 +0x185

goroutine 77 [select]:
github.com/cockroachdb/cockroach/pkg/util/admission.makeWorkQueue.func1(0xc0008bb720, 0xc00010f6e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/work_queue.go:217 +0x9c
created by github.com/cockroachdb/cockroach/pkg/util/admission.makeWorkQueue
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/work_queue.go:213 +0x185

goroutine 78 [select]:
github.com/cockroachdb/cockroach/pkg/util/admission.makeWorkQueue.func1(0xc0008bb7c0, 0xc000274180)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/work_queue.go:217 +0x9c
created by github.com/cockroachdb/cockroach/pkg/util/admission.makeWorkQueue
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/work_queue.go:213 +0x185

goroutine 79 [select]:
github.com/cockroachdb/cockroach/pkg/util/admission.makeWorkQueue.func1(0xc0008bb860, 0xc0003497a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/work_queue.go:217 +0x9c
created by github.com/cockroachdb/cockroach/pkg/util/admission.makeWorkQueue
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/work_queue.go:213 +0x185

goroutine 80 [select]:
github.com/cockroachdb/cockroach/pkg/util/admission.makeWorkQueue.func1(0xc0008bb900, 0xc000378b40)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/work_queue.go:217 +0x9c
created by github.com/cockroachdb/cockroach/pkg/util/admission.makeWorkQueue
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/work_queue.go:213 +0x185

goroutine 81 [chan receive, 5 minutes]:
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x688b290, 0xc000b18b70)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:293 +0x76
runtime/pprof.Do(0x688b220, 0xc000b18b70, 0xc00007e5a0, 0x1, 0x1, 0xc0006117b8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcf
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc0003a6000)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:292 +0x89
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:284 +0xa8

goroutine 82 [chan receive, 5 minutes]:
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x688b290, 0xc000b18a20)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:293 +0x76
runtime/pprof.Do(0x688b220, 0xc000b18a20, 0xc00007e5a0, 0x1, 0x1, 0xc0010b07b8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcf
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc0003a65a0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:292 +0x89
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:284 +0xa8

goroutine 83 [chan receive, 5 minutes]:
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x688b290, 0xc000b18c30)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:293 +0x76
runtime/pprof.Do(0x688b220, 0xc000b18c30, 0xc00007e5a0, 0x1, 0x1, 0xc0010b0fb8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcf
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc0003a66e0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:292 +0x89
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:284 +0xa8

goroutine 84 [chan receive, 5 minutes]:
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x688b290, 0xc000b18c90)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:293 +0x76
runtime/pprof.Do(0x688b220, 0xc000b18c90, 0xc00007e5a0, 0x1, 0x1, 0xc0010b17b8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcf
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc0003a6c80)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:292 +0x89
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:284 +0xa8

goroutine 85 [sync.Cond.Wait, 5 minutes]:
sync.runtime_notifyListWait(0xc000024fe0, 0x0)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc000024fd0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/pebble/record.(*flusherCond).Wait(...)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/record/log_writer.go:193
github.com/cockroachdb/pebble/record.(*LogWriter).flushLoop(0xc000024f00, 0x688b290, 0xc000ad2840)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/record/log_writer.go:396 +0x10b
runtime/pprof.Do(0x688b220, 0xc000ad2840, 0xc00007e4a0, 0x1, 0x1, 0xc0010b1fb8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcf
github.com/cockroachdb/pebble/record.NewLogWriter.func2(0xc000024f00)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/record/log_writer.go:314 +0x89
created by github.com/cockroachdb/pebble/record.NewLogWriter
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/record/log_writer.go:313 +0x2ea

goroutine 53 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/gossip.(*server).start.func3(0x688b290, 0xc0007a2570)
	/go/src/github.com/cockroachdb/cockroach/pkg/gossip/server.go:380 +0x5a
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0007a2570, 0xc000c30b80, 0x0, 0xc0018e6140)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 22 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2a80)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2a80, 0xc000c30b80, 0x0, 0xc0004dad60)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 23 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2ab0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2ab0, 0xc000c30b80, 0x0, 0xc0004dad70)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 24 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2ae0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2ae0, 0xc000c30b80, 0x0, 0xc0004dad80)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 25 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2b10)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2b10, 0xc000c30b80, 0x0, 0xc0004dad90)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 26 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2b40)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2b40, 0xc000c30b80, 0x0, 0xc0004dada0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 27 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2b70)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2b70, 0xc000c30b80, 0x0, 0xc0004dadb0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 28 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2ba0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2ba0, 0xc000c30b80, 0x0, 0xc0004dadc0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 29 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2bd0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2bd0, 0xc000c30b80, 0x0, 0xc0004dadd0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 30 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2c00)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2c00, 0xc000c30b80, 0x0, 0xc0004dade0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 31 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2c30)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2c30, 0xc000c30b80, 0x0, 0xc0004dadf0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 32 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2c60)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2c60, 0xc000c30b80, 0x0, 0xc0004dae00)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 33 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2c90)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2c90, 0xc000c30b80, 0x0, 0xc0004dae10)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 98 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2cc0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2cc0, 0xc000c30b80, 0x0, 0xc0004dae20)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 99 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2cf0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2cf0, 0xc000c30b80, 0x0, 0xc0004dae30)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 100 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x688b290, 0xc0004e2d20)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:108 +0x11e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2d20, 0xc000c30b80, 0x0, 0xc0004dae40)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 101 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initCancelingWorkers.func1(0x688b290, 0xc0004e2d50)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:134 +0xc7
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2d50, 0xc000c30b80, 0x0, 0xc0004d74d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 102 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initCancelingWorkers.func1(0x688b290, 0xc0004e2d80)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:134 +0xc7
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2d80, 0xc000c30b80, 0x0, 0xc0004d74e8)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 103 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initCancelingWorkers.func1(0x688b290, 0xc0004e2db0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:134 +0xc7
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2db0, 0xc000c30b80, 0x0, 0xc0004d7500)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 104 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initCancelingWorkers.func1(0x688b290, 0xc0004e2de0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:134 +0xc7
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2de0, 0xc000c30b80, 0x0, 0xc0004d7518)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 105 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).processEvents(0xc0004de400, 0x688b290, 0xc0004e2f30, 0xc0004e2e40, 0xc000487200, 0xc0004872c0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:347 +0xed
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).run(0xc0004de400, 0x688b290, 0xc0004e2f30, 0xc0004e2e40)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:257 +0x605
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).Start.func1(0x688b290, 0xc0004e2f30)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:184 +0x47
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2f30, 0xc000c30b80, 0x0, 0xc0004d7590)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 106 [semacquire, 4 minutes]:
sync.runtime_Semacquire(0xc0004e30f0)
	/usr/local/go/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc0004e30e8)
	/usr/local/go/src/sync/waitgroup.go:130 +0x65
golang.org/x/sync/errgroup.(*Group).Wait(0xc0004e30e0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:40 +0x31
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait(0xc0004e30e0, 0x688b1e8, 0xc0004d3a40, 0xc001a46660, 0xc0004d3a40)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:142 +0x57
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed(0xc00080c000, 0x688b290, 0xc0004e2f30, 0xc000868118, 0x1, 0x8, 0xc000868170, 0x1, 0x1, 0x16b6985cf71ccfc9, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:93 +0x4f8
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*dbAdapter).RangeFeed(0xc000d30b70, 0x688b290, 0xc0004e2f30, 0xc000868118, 0x1, 0x8, 0xc000868170, 0x1, 0x1, 0x16b6985cf71ccfc9, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/db_adapter.go:69 +0xb4
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).run.func1(0x688b290, 0xc0004e2f30)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:251 +0xe6
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e2f30, 0xc000c30b80, 0x0, 0xc0004e3050)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 107 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed.func1(0x688b1e8, 0xc0004d3a40, 0xc000a5c5a0, 0x5776f70)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:76 +0x145
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc0004e30e0, 0xc0004e3110)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 1119 [select]:
github.com/cockroachdb/cockroach/pkg/rpc.respStreamClientAdapter.recvInternal(0x688b1e8, 0xc002081a00, 0xc002670720, 0xc002670780, 0xc001a5f8c8, 0x2, 0x2, 0xc001a5f718)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:560 +0xb4
github.com/cockroachdb/cockroach/pkg/rpc.rangeFeedClientAdapter.Recv(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:589
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).singleRangeFeed(0xc00080c000, 0x688b1e8, 0xc0004d3a40, 0xc000868118, 0x1, 0x8, 0xc000868170, 0x1, 0x1, 0x16b6985cf71ccfc9, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:401 +0x959
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).partialRangeFeed(0xc00080c000, 0x688b1e8, 0xc0004d3a40, 0xc001a40360, 0xc000868118, 0x1, 0x8, 0xc000868170, 0x1, 0x1, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:277 +0x4a9
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed.func1.1(0x688b1e8, 0xc0004d3a40, 0xc000a5c5a0, 0x5776f70)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:80 +0x145
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc0004e30e0, 0xc00269d5c0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 1120 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).rangeFeedWithRangeID(0xc0019f6a80, 0x10, 0xc00269b300, 0x68e13e8, 0xc002696480, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_rangefeed.go:244 +0x670
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).RangeFeed(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_rangefeed.go:142
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).RangeFeed(0xc00026c800, 0xc00269b300, 0x68e13e8, 0xc002696480, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:2661 +0x136
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).RangeFeed(0xc0002d02a0, 0xc00269b300, 0x68e13e8, 0xc002696480, 0xc000091768)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/stores.go:216 +0xce
github.com/cockroachdb/cockroach/pkg/server.(*Node).RangeFeed(0xc000238a00, 0xc00269b300, 0x68e13e8, 0xc002696480, 0xc000091768, 0x1ab3fba)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:1090 +0x54
github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.RangeFeed.func1(0xc001f53c40, 0x68f1c78, 0xc000238a00, 0xc00269b300, 0x688b1e8, 0xc002081a00, 0xc002670720, 0xc002670780)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:617 +0xc5
created by github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.RangeFeed
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:615 +0x12e

goroutine 1197 [IO wait]:
internal/poll.runtime_pollWait(0x7f32aa3eac10, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc002ccd118, 0x72, 0x700, 0x752, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc002ccd100, 0xc002446000, 0x752, 0x752, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc002ccd100, 0xc002446000, 0x752, 0x752, 0x0, 0x67f5a80, 0x91f6a60)
	/usr/local/go/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc002af81d8, 0xc002446000, 0x752, 0x752, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:183 +0x91
crypto/tls.(*atLeastReader).Read(0xc0025f5848, 0xc002446000, 0x752, 0x752, 0x8e59700, 0xc000300400, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:776 +0x63
bytes.(*Buffer).ReadFrom(0xc0029c8278, 0x67eb580, 0xc0025f5848, 0x4376e5, 0x49b90e0, 0x4fc4ec0)
	/usr/local/go/src/bytes/buffer.go:204 +0xbe
crypto/tls.(*Conn).readFromUntil(0xc0029c8000, 0x67f5d60, 0xc002af81d8, 0x5, 0xc002af81d8, 0x4ab6120)
	/usr/local/go/src/crypto/tls/conn.go:798 +0xf3
crypto/tls.(*Conn).readRecordOrCCS(0xc0029c8000, 0x0, 0x0, 0x873a11)
	/usr/local/go/src/crypto/tls/conn.go:605 +0x115
crypto/tls.(*Conn).readRecord(...)
	/usr/local/go/src/crypto/tls/conn.go:573
crypto/tls.(*Conn).Read(0xc0029c8000, 0xc002ebb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1276 +0x165
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*readTimeoutConn).Read(0xc002756a98, 0xc002ebb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:1761 +0x12a
bufio.(*Reader).fill(0xc001f6eaf0)
	/usr/local/go/src/bufio/bufio.go:101 +0x108
bufio.(*Reader).ReadByte(0xc001f6eaf0, 0xd3a1ca6, 0xed91f7601, 0x0)
	/usr/local/go/src/bufio/bufio.go:253 +0x39
github.com/cockroachdb/cockroach/pkg/sql/pgwire/pgwirebase.(*ReadBuffer).ReadTypedMsg(0xc001f6f2c0, 0x684dcd0, 0xc001f6eaf0, 0x6845258, 0x91f7338, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/pgwirebase/encoding.go:203 +0x35
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).serveImpl.func5(0xc001f6ea80, 0xc002ea6bf0, 0xc0025056b2, 0xc0025056b3, 0x685cf70, 0xc002ccd180, 0xc001a54700, 0xc002e85bcc, 0xc0025056b1, 0x100, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:362 +0x71
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).serveImpl(0xc001f6ea80, 0x688b290, 0xc0008b1260, 0xc002ea6be0, 0xc001a54700, 0x5400, 0x15000, 0xc0004e0c80, 0x0, 0x4, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:496 +0x638
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*Server).serveConn(0xc00199a100, 0x688b290, 0xc0008b1260, 0x68eb178, 0xc0029c8000, 0xc000c23014, 0x4, 0x1, 0xc0008b1200, 0xc0008b1230, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:169 +0x218
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*Server).ServeConn(0xc00199a100, 0x688b290, 0xc0008b1260, 0x68eff28, 0xc0029c8000, 0x688b201, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/server.go:700 +0xe78
github.com/cockroachdb/cockroach/pkg/server.(*SQLServer).startServeSQL.func1.1(0x688b290, 0xc002dee5a0, 0x68eff28, 0xc002af81d8)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2385 +0xff
github.com/cockroachdb/cockroach/pkg/util/netutil.(*Server).ServeWith.func1(0x688b290, 0xc002dee5a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:162 +0x126
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002dee5a0, 0xc000c30b80, 0x0, 0xc002cd3d10)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 87 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/server.startPurgeOldSessions.func1(0x688b290, 0xc0019948d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/purge_auth_session.go:66 +0x225
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0019948d0, 0xc000c30b80, 0x0, 0xc001984120)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 88 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/util/hlc.(*Clock).StartMonitoringForwardClockJumps.func1(0x577b5b8, 0xc00065e8c0, 0xc000654540, 0x688b290, 0xc0008b1a40, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/hlc/hlc.go:247 +0xff
created by github.com/cockroachdb/cockroach/pkg/util/hlc.(*Clock).StartMonitoringForwardClockJumps
	/go/src/github.com/cockroachdb/cockroach/pkg/util/hlc/hlc.go:240 +0xd0

goroutine 89 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/util/netutil.MakeServer.func2(0x688b290, 0xc001994a50)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:113 +0x65
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001994a50, 0xc000c30b80, 0x0, 0xc0019841a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 90 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/server.(*Server).startServeUI.func1(0x688b290, 0xc001994b40)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2310 +0x53
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001994b40, 0xc000c30b80, 0x0, 0xc00198a198)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 91 [IO wait, 4 minutes]:
internal/poll.runtime_pollWait(0x7f32aa3eafb0, 0x72, 0x0)
	/usr/local/go/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc0006fc298, 0x72, 0x0, 0x0, 0x50c4331)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc0006fc280, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:401 +0x212
net.(*netFD).accept(0xc0006fc280, 0xc, 0x7f32d4934f18, 0x10)
	/usr/local/go/src/net/fd_unix.go:172 +0x45
net.(*TCPListener).accept(0xc00198a180, 0x43a0b8, 0xc, 0x4b1db60)
	/usr/local/go/src/net/tcpsock_posix.go:139 +0x32
net.(*TCPListener).Accept(0xc00198a180, 0xc000c2e000, 0xc0000e07e0, 0xc000486720, 0xc0000e07e0)
	/usr/local/go/src/net/tcpsock.go:261 +0x65
github.com/cockroachdb/cmux.(*cMux).Serve(0xc00199c1c0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux/cmux.go:124 +0x96
github.com/cockroachdb/cockroach/pkg/server.(*Server).startServeUI.func2(0x688b290, 0xc001994ba0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2327 +0x2f
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001994ba0, 0xc000c30b80, 0x0, 0xc00198a1e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 92 [chan receive, 4 minutes]:
github.com/cockroachdb/cmux.muxListener.Accept(...)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux/cmux.go:184
net/http.(*Server).Serve(0xc0018d6000, 0x6867590, 0xc00198a1b0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:2961 +0x285
github.com/cockroachdb/cockroach/pkg/server.(*Server).startServeUI.func3(0x688b290, 0xc001994c00)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2345 +0x127
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001994c00, 0xc000c30b80, 0x0, 0xc001994bd0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 93 [chan receive, 4 minutes]:
github.com/cockroachdb/cmux.muxListener.Accept(...)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux/cmux.go:184
crypto/tls.(*listener).Accept(0xc00198a1f8, 0x4d9eae0, 0xc0018fe090, 0x49044e0, 0x8dcab80)
	/usr/local/go/src/crypto/tls/tls.go:67 +0x37
net/http.(*Server).Serve(0xc0004faee0, 0x684de20, 0xc00198a1f8, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:2961 +0x285
github.com/cockroachdb/cockroach/pkg/server.(*Server).startServeUI.func4(0x688b290, 0xc001994c30)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2358 +0x3f
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001994c30, 0xc000c30b80, 0x0, 0xc00198a210)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 43 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/util/netutil.MakeServer.func2(0x688b290, 0xc0004e3740)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:113 +0x65
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0004e3740, 0xc000c30b80, 0x0, 0xc0004d5a40)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 94 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/server.StartListenRPCAndSQL.func1(0x688b290, 0xc001994ff0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2211 +0x4c
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001994ff0, 0xc000c30b80, 0x0, 0xc00198a2d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 95 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/server.StartListenRPCAndSQL.func3(0x688b290, 0xc001995080)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2255 +0x4d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001995080, 0xc000c30b80, 0x0, 0xc001984340)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 96 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/server.ConfigureGRPCGateway.func1(0x688b290, 0xc001995260)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1949 +0x45
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001995260, 0xc000c30b80, 0x0, 0xc00198a300)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 97 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/server.(*loopbackListener).Accept(0xc001995230, 0x5779330, 0xc000adcfc0, 0x6855ef0, 0xc001995230)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/loopback.go:49 +0xb5
google.golang.org/grpc.(*Server).Serve(0xc000adcfc0, 0x6855ef0, 0xc001995230, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/server.go:786 +0x27f
github.com/cockroachdb/cockroach/pkg/server.ConfigureGRPCGateway.func2(0x688b290, 0xc001995290)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1957 +0x45
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001995290, 0xc000c30b80, 0x0, 0xc00198a318)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 114 [select, 4 minutes]:
google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc0019802d0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:71 +0xb2
created by google.golang.org/grpc.newCCBalancerWrapper
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:62 +0x1de

goroutine 115 [chan receive, 4 minutes]:
google.golang.org/grpc.(*addrConn).resetTransport(0xc001a21340)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:1219 +0x465
created by google.golang.org/grpc.(*addrConn).connect
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:849 +0x12a

goroutine 116 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/server.ConfigureGRPCGateway.func5(0x688b290, 0xc001995470)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1990 +0x4c
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001995470, 0xc000c30b80, 0x0, 0xc00198a378)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 46 [select]:
net.(*pipe).read(0xc0004dca00, 0xc000143b00, 0x8d8, 0x8d8, 0xc000143b05, 0x12, 0x8d3)
	/usr/local/go/src/net/pipe.go:159 +0x185
net.(*pipe).Read(0xc0004dca00, 0xc000143b00, 0x8d8, 0x8d8, 0x43995b, 0xc002433350, 0x18)
	/usr/local/go/src/net/pipe.go:142 +0x56
crypto/tls.(*atLeastReader).Read(0xc002433350, 0xc000143b00, 0x8d8, 0x8d8, 0x8d3, 0xc000300000, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:776 +0x63
bytes.(*Buffer).ReadFrom(0xc000cb33f8, 0x67eb580, 0xc002433350, 0x4376e5, 0x49b90e0, 0x4dee5a0)
	/usr/local/go/src/bytes/buffer.go:204 +0xbe
crypto/tls.(*Conn).readFromUntil(0xc000cb3180, 0x7f32aa2745b8, 0xc0004dca00, 0x5, 0xc0004dca00, 0x11)
	/usr/local/go/src/crypto/tls/conn.go:798 +0xf3
crypto/tls.(*Conn).readRecordOrCCS(0xc000cb3180, 0x0, 0x0, 0xdfd43e)
	/usr/local/go/src/crypto/tls/conn.go:605 +0x115
crypto/tls.(*Conn).readRecord(...)
	/usr/local/go/src/crypto/tls/conn.go:573
crypto/tls.(*Conn).Read(0xc000cb3180, 0xc001956000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1276 +0x165
bufio.(*Reader).Read(0xc00193af60, 0xc0018d6658, 0x9, 0x9, 0x14, 0xc001350780, 0xc001b62cf0)
	/usr/local/go/src/bufio/bufio.go:227 +0x222
io.ReadAtLeast(0x67eb320, 0xc00193af60, 0xc0018d6658, 0x9, 0x9, 0x9, 0x7f32d4934f18, 0x18, 0xc002433338)
	/usr/local/go/src/io/io.go:328 +0x87
io.ReadFull(...)
	/usr/local/go/src/io/io.go:347
golang.org/x/net/http2.readFrameHeader(0xc0018d6658, 0x9, 0x9, 0x67eb320, 0xc00193af60, 0x0, 0x0, 0xc0000e1620, 0xc001b62d57)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2/frame.go:237 +0x89
golang.org/x/net/http2.(*Framer).ReadFrame(0xc0018d6620, 0xc004ebdea8, 0xc004ebdea8, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2/frame.go:492 +0xa5
google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc00000c1e0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1495 +0x1a5
created by google.golang.org/grpc/internal/transport.newHTTP2Client
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:355 +0xdd1

goroutine 117 [chan receive, 4 minutes]:
github.com/cockroachdb/cmux.muxListener.Accept(...)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux/cmux.go:184
google.golang.org/grpc.(*Server).Serve(0xc000adcfc0, 0x6867590, 0xc00198a2e8, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/server.go:786 +0x27f
github.com/cockroachdb/cockroach/pkg/server.StartListenRPCAndSQL.func5.1(0x688b290, 0xc000ad3c80)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2281 +0x3f
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc000ad3c80, 0xc000c30b80, 0x0, 0xc001985180)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 118 [IO wait, 4 minutes]:
internal/poll.runtime_pollWait(0x7f32aa3eaec8, 0x72, 0x0)
	/usr/local/go/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc0006fc518, 0x72, 0x0, 0x0, 0x50c4331)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc0006fc500, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:401 +0x212
net.(*netFD).accept(0xc0006fc500, 0x37e11d600, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:172 +0x45
net.(*TCPListener).accept(0xc00198a2a0, 0xc000b3ae00, 0xc000b3ae08, 0x28)
	/usr/local/go/src/net/tcpsock_posix.go:139 +0x32
net.(*TCPListener).Accept(0xc00198a2a0, 0x5770748, 0xc00199c240, 0x68eff28, 0xc002c4fe48)
	/usr/local/go/src/net/tcpsock.go:261 +0x65
github.com/cockroachdb/cmux.(*cMux).Serve(0xc00199c240, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux/cmux.go:124 +0x96
github.com/cockroachdb/cockroach/pkg/server.StartListenRPCAndSQL.func5.2.1()
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2286 +0x2f
sync.(*Once).doSlow(0xc000869420, 0xc0006ef708)
	/usr/local/go/src/sync/once.go:68 +0xec
sync.(*Once).Do(...)
	/usr/local/go/src/sync/once.go:59
github.com/cockroachdb/cockroach/pkg/server.StartListenRPCAndSQL.func5.2(0x688b290, 0xc000ad3e00)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2285 +0x74
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc000ad3e00, 0xc000c30b80, 0x0, 0xc0019851a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 119 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/server.(*Server).refreshSettings.func1(0x688b290, 0xc000c17140)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/settingsworker.go:78 +0xc5
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc000c17140, 0xc000c30b80, 0x0, 0xc001987010)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 132 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2(0x688b290, 0xc000ae1ef0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:810 +0x125
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc000ae1ef0, 0xc000c30b80, 0x0, 0xc00076a580)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 47 [select]:
google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0013521e0, 0x1, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:406 +0xff
google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc00193b4a0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:533 +0x1dd
google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc00000c1e0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:405 +0x7b
created by google.golang.org/grpc/internal/transport.newHTTP2Client
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:403 +0x12ae

goroutine 48 [select]:
google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc001352870, 0x1, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:406 +0xff
google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc00193b500, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:533 +0x1dd
google.golang.org/grpc/internal/transport.NewServerTransport.func2(0xc001350600)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_server.go:304 +0xd7
created by google.golang.org/grpc/internal/transport.NewServerTransport
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_server.go:301 +0x11b0

goroutine 49 [select]:
google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc001350600)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_server.go:1061 +0x265
created by google.golang.org/grpc/internal/transport.NewServerTransport
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_server.go:313 +0x11d5

goroutine 130 [select]:
net.(*pipe).read(0xc0004dca80, 0xc001954000, 0x796, 0x796, 0xc001954005, 0x12, 0x791)
	/usr/local/go/src/net/pipe.go:159 +0x185
net.(*pipe).Read(0xc0004dca80, 0xc001954000, 0x796, 0x796, 0x43995b, 0xc002433368, 0x18)
	/usr/local/go/src/net/pipe.go:142 +0x56
crypto/tls.(*atLeastReader).Read(0xc002433368, 0xc001954000, 0x796, 0x796, 0x791, 0xc000300000, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:776 +0x63
bytes.(*Buffer).ReadFrom(0xc000c393f8, 0x67eb580, 0xc002433368, 0x4376e5, 0x49b90e0, 0x4dee5a0)
	/usr/local/go/src/bytes/buffer.go:204 +0xbe
crypto/tls.(*Conn).readFromUntil(0xc000c39180, 0x7f32aa2745b8, 0xc0004dca80, 0x5, 0xc0004dca80, 0x11)
	/usr/local/go/src/crypto/tls/conn.go:798 +0xf3
crypto/tls.(*Conn).readRecordOrCCS(0xc000c39180, 0x0, 0x0, 0xdfd43e)
	/usr/local/go/src/crypto/tls/conn.go:605 +0x115
crypto/tls.(*Conn).readRecord(...)
	/usr/local/go/src/crypto/tls/conn.go:573
crypto/tls.(*Conn).Read(0xc000c39180, 0xc001978000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1276 +0x165
bufio.(*Reader).Read(0xc00193b3e0, 0xc0018d6b98, 0x9, 0x9, 0x14, 0x7f32d4934f18, 0x18)
	/usr/local/go/src/bufio/bufio.go:227 +0x222
io.ReadAtLeast(0x67eb320, 0xc00193b3e0, 0xc0018d6b98, 0x9, 0x9, 0x9, 0xdcf5c5, 0xc004ebdecc, 0xc001849b60)
	/usr/local/go/src/io/io.go:328 +0x87
io.ReadFull(...)
	/usr/local/go/src/io/io.go:347
golang.org/x/net/http2.readFrameHeader(0xc0018d6b98, 0x9, 0x9, 0x67eb320, 0xc00193b3e0, 0x0, 0x0, 0xc004ebdec0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2/frame.go:237 +0x89
golang.org/x/net/http2.(*Framer).ReadFrame(0xc0018d6b60, 0xc004ebdec0, 0x91a1b40, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2/frame.go:492 +0xa5
google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc001350600, 0xc000ae0bd0, 0x5779370)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_server.go:541 +0x9b
google.golang.org/grpc.(*Server).serveStreams(0xc000adcfc0, 0x68f45b8, 0xc001350600)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/server.go:925 +0x103
google.golang.org/grpc.(*Server).handleRawConn.func1(0xc000adcfc0, 0x68f45b8, 0xc001350600, 0x50c7bd8, 0x8)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/server.go:875 +0x3f
created by google.golang.org/grpc.(*Server).handleRawConn
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/server.go:874 +0x5f4

goroutine 54 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/gossip.(*Gossip).bootstrap.func1(0x688b290, 0xc000c17800)
	/go/src/github.com/cockroachdb/cockroach/pkg/gossip/gossip.go:1326 +0x325
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0007a2720, 0xc000c30b80, 0x0, 0xc00079c270)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 55 [select]:
github.com/cockroachdb/cockroach/pkg/gossip.(*Gossip).manage.func1(0x688b290, 0xc0007a2750)
	/go/src/github.com/cockroachdb/cockroach/pkg/gossip/gossip.go:1361 +0x30a
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0007a2750, 0xc000c30b80, 0x0, 0xc00079c280)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 56 [select, 3 minutes]:
github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher.(*RequestBatcher).run(0xc0018e4000, 0x688b290, 0xc000abc0f0)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher/batcher.go:466 +0x33e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc000abc0f0, 0xc000c30b80, 0x0, 0xc00079d790)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 57 [select, 3 minutes]:
github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher.(*RequestBatcher).run(0xc0018e4120, 0x688b290, 0xc000abc180)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher/batcher.go:466 +0x33e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc000abc180, 0xc000c30b80, 0x0, 0xc00079d7a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 58 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher.(*RequestBatcher).run(0xc0018e4240, 0x688b290, 0xc000abc1e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher/batcher.go:466 +0x33e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc000abc1e0, 0xc000c30b80, 0x0, 0xc00079d7b0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 59 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).Start.func1(0x688b290, 0xc0017d7b60)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:193 +0x4c
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7b60, 0xc000c30b80, 0x0, 0xc0013493e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 60 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc00000058b)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7b90)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7b90, 0xc000c30b80, 0x0, 0xc000970bf0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 61 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000581)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7bc0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7bc0, 0xc000c30b80, 0x0, 0xc000970c00)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 62 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000588)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7bf0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7bf0, 0xc000c30b80, 0x0, 0xc000970c10)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 63 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000584)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7c50)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7c50, 0xc000c30b80, 0x0, 0xc000970c20)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 64 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc00000058a)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7c80)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7c80, 0xc000c30b80, 0x0, 0xc000970c30)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 65 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc00000058e)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7cb0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7cb0, 0xc000c30b80, 0x0, 0xc000970c40)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 146 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000598)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7ce0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7ce0, 0xc000c30b80, 0x0, 0xc000970c50)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 147 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000597)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7d40)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7d40, 0xc000c30b80, 0x0, 0xc000970c60)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 148 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000593)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7d70)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7d70, 0xc000c30b80, 0x0, 0xc000970d10)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 149 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc00000059f)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7da0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7da0, 0xc000c30b80, 0x0, 0xc000970d20)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 150 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc00000059b)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7dd0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7dd0, 0xc000c30b80, 0x0, 0xc000970d30)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 151 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc00000059e)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7e30)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7e30, 0xc000c30b80, 0x0, 0xc000970d40)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 152 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc00000059d)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7e60)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7e60, 0xc000c30b80, 0x0, 0xc000970d50)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 153 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000585)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7e90)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7e90, 0xc000c30b80, 0x0, 0xc000970d60)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 154 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000592)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7ec0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7ec0, 0xc000c30b80, 0x0, 0xc000970d70)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 155 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000586)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7ef0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7ef0, 0xc000c30b80, 0x0, 0xc000970d80)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 156 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000582)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7f50)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7f50, 0xc000c30b80, 0x0, 0xc000970d90)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 157 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000590)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7f80)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7f80, 0xc000c30b80, 0x0, 0xc000970da0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 158 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc00000058c)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0017d7fb0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0017d7fb0, 0xc000c30b80, 0x0, 0xc000970db0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 159 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000587)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e4000)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4000, 0xc000c30b80, 0x0, 0xc000970dc0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 160 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000595)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e4030)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4030, 0xc000c30b80, 0x0, 0xc000970dd0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 161 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000589)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e4060)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4060, 0xc000c30b80, 0x0, 0xc000970de0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 162 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000594)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e4090)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4090, 0xc000c30b80, 0x0, 0xc000970df0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 163 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc00000058d)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e40c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e40c0, 0xc000c30b80, 0x0, 0xc000970e70)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 164 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc00000059c)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e40f0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e40f0, 0xc000c30b80, 0x0, 0xc000970e80)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 165 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000591)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e4120)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4120, 0xc000c30b80, 0x0, 0xc000970e90)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 166 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc00000059a)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e4150)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4150, 0xc000c30b80, 0x0, 0xc000970ea0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 167 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000596)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e4180)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4180, 0xc000c30b80, 0x0, 0xc000970eb0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 168 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc0000005a0)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e41b0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e41b0, 0xc000c30b80, 0x0, 0xc000970ec0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 169 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc00000058f)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e41e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e41e0, 0xc000c30b80, 0x0, 0xc000970ed0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 170 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000599)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e4210)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4210, 0xc000c30b80, 0x0, 0xc000970ee0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 171 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0006638d0, 0xc000000583)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0006638c0)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc001038000, 0x688b290, 0xc0015e4240)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:265 +0x8b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4240, 0xc000c30b80, 0x0, 0xc000970ef0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 172 [semacquire, 4 minutes]:
sync.runtime_Semacquire(0xc001038090)
	/usr/local/go/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc001038088)
	/usr/local/go/src/sync/waitgroup.go:130 +0x65
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).Wait(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:226
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4270, 0xc000c30b80, 0x0, 0xc000970f00)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 173 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).raftTickLoop(0xc00026c800, 0x688b290, 0xc0015e42a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store_raft.go:618 +0x232
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e42a0, 0xc000c30b80, 0x0, 0xc000970f10)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 174 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).coalescedHeartbeatsLoop(0xc00026c800, 0x688b290, 0xc0015e42d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store_raft.go:678 +0x10e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e42d0, 0xc000c30b80, 0x0, 0xc000970f20)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 175 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).Start.func2(0x688b290, 0xc0015e4300)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:1612 +0xb3
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4300, 0xc000c30b80, 0x0, 0xc0013493f8)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 176 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).startGossip.func3(0x688b290, 0xc0015e4360)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:1777 +0x291
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4360, 0xc000c30b80, 0x0, 0xc0002a2eb0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 177 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).startGossip.func3(0x688b290, 0xc0015e4390)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:1777 +0x291
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4390, 0xc000c30b80, 0x0, 0xc0002a2f50)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 178 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).startGossip.func3(0x688b290, 0xc0015e43c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:1777 +0x291
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e43c0, 0xc000c30b80, 0x0, 0xc0002a3040)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 180 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).startLeaseRenewer.func1(0x688b290, 0xc0015e4420)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:1855 +0x2b0
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4420, 0xc000c30b80, 0x0, 0xc000970fd0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 181 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).startRangefeedUpdater.func1(0x688b290, 0xc0015e4450)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:1903 +0x2b5
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4450, 0xc000c30b80, 0x0, 0xc000970fe0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 182 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*StoreRebalancer).Start.func1(0x688b290, 0xc0015e44e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store_rebalancer.go:185 +0x16d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e44e0, 0xc000c30b80, 0x0, 0xc001349428)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 183 [select]:
github.com/cockroachdb/cockroach/pkg/server.(*Node).startComputePeriodicMetrics.func1(0x688b290, 0xc0015e47b0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:698 +0x105
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e47b0, 0xc000c30b80, 0x0, 0xc0018e6b20)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 184 [select]:
github.com/cockroachdb/cockroach/pkg/server.(*Node).startGossiping.func1(0x688b290, 0xc0015e47e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:663 +0x1df
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e47e0, 0xc000c30b80, 0x0, 0xc001349440)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 185 [select]:
github.com/cockroachdb/cockroach/pkg/util/admission.makeWorkQueue.func1(0xc00159c780, 0xc0019d7200)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/work_queue.go:217 +0x9c
created by github.com/cockroachdb/cockroach/pkg/util/admission.makeWorkQueue
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/work_queue.go:213 +0x185

goroutine 186 [select]:
github.com/cockroachdb/cockroach/pkg/util/admission.(*StoreGrantCoordinators).SetPebbleMetricsProvider.func1(0xc000b2db20)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/granter.go:1165 +0xda
created by github.com/cockroachdb/cockroach/pkg/util/admission.(*StoreGrantCoordinators).SetPebbleMetricsProvider
	/go/src/github.com/cockroachdb/cockroach/pkg/util/admission/granter.go:1160 +0x24e

goroutine 187 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/server.periodicallyPersistHLCUpperBound(0xc00065e8c0, 0xc00017e150, 0xc000971160, 0x577b5b8, 0xc0000e07e0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1111 +0x150
github.com/cockroachdb/cockroach/pkg/server.(*Server).startPersistingHLCUpperBound.func2(0x688b290, 0xc0015e49c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1187 +0x69
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e49c0, 0xc000c30b80, 0x0, 0xc0015e4990)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 188 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/reports.(*Reporter).Start.func2(0x688b290, 0xc0015e49f0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/reports/reporter.go:158 +0x1cd
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e49f0, 0xc000c30b80, 0x0, 0xc001349488)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 189 [select]:
github.com/cockroachdb/cockroach/pkg/server.startSampleEnvironment.func1(0x688b290, 0xc0015e4a20)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2615 +0x1d2
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4a20, 0xc000c30b80, 0x0, 0xc00167a700)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 190 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/liveness.(*NodeLiveness).Start.func1(0x688b290, 0xc0015e4c90)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/liveness/liveness.go:774 +0x43d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e4c90, 0xc000c30b80, 0x0, 0xc00017c070)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 391 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).Start.func8(0x688b290, 0xc002153ad0)
	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:872 +0x1d9
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002153ad0, 0xc000c30b80, 0x0, 0xc0013de900)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 313 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).addToPurgatoryLocked.func2(0x688b290, 0xc00155a8d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:1152 +0x30d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc00155a8d0, 0xc000c30b80, 0x0, 0xc00069fea8)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 133 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2(0x688b290, 0xc000ae1f20)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:810 +0x125
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc000ae1f20, 0xc000c30b80, 0x0, 0xc00076a5a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 134 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2(0x688b290, 0xc000ae1f80)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:810 +0x125
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc000ae1f80, 0xc000c30b80, 0x0, 0xc00076a5e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 135 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2(0x688b290, 0xc000ae1fb0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:810 +0x125
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc000ae1fb0, 0xc000c30b80, 0x0, 0xc00076a600)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 136 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2(0x688b290, 0xc00151a000)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:810 +0x125
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc00151a000, 0xc000c30b80, 0x0, 0xc00076a620)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 137 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2(0x688b290, 0xc00151a030)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:810 +0x125
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc00151a030, 0xc000c30b80, 0x0, 0xc00076a680)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 138 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2(0x688b290, 0xc00151a060)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:810 +0x125
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc00151a060, 0xc000c30b80, 0x0, 0xc00076a700)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 139 [select, 3 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2(0x688b290, 0xc00151a090)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:810 +0x125
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc00151a090, 0xc000c30b80, 0x0, 0xc00076a720)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 140 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2(0x688b290, 0xc00151a0c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:810 +0x125
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc00151a0c0, 0xc000c30b80, 0x0, 0xc00076a740)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 141 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaScanner).waitAndProcess(0xc0007401a0, 0x688b290, 0xc00151a0f0, 0x88ad890, 0xed91f7717, 0x0, 0xc001514a80, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scanner.go:219 +0x19d
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaScanner).scanLoop.func1.1(0xc001514a80, 0xc00047b401)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scanner.go:281 +0x78
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*storeReplicaVisitor).Visit(0xc00047b4d0, 0xc003176300)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:396 +0x151
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaScanner).scanLoop.func1(0x688b290, 0xc00151a0f0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scanner.go:279 +0x19b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc00151a0f0, 0xc000c30b80, 0x0, 0xc0004dba70)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 1175 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run(0xc0014df760, 0x688b290, 0xc0026fc360, 0x10, 0xc0026fc330, 0xc000a5c5a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:249 +0x3f6
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).Start.func1(0x688b290, 0xc0026fc360)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:204 +0x59
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0026fc360, 0xc000c30b80, 0x0, 0xc002696700)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 2269 [select]:
google.golang.org/grpc/internal/transport.(*http2Client).keepalive(0xc002d8b860)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1569 +0x17a
created by google.golang.org/grpc/internal/transport.newHTTP2Client
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:350 +0x178b

goroutine 210 [select]:
github.com/cockroachdb/cockroach/pkg/server.(*Node).startWriteNodeStatus.func1(0x688b290, 0xc0015e5b00)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:774 +0xef
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e5b00, 0xc000c30b80, 0x0, 0xc000e3a0d8)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 211 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/protectedts/ptcache.(*Cache).periodicallyRefreshProtectedtsCache(0xc0003ea8f0, 0x688b290, 0xc0015e5b30)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/protectedts/ptcache/cache.go:160 +0x239
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e5b30, 0xc000c30b80, 0x0, 0xc000978130)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 212 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/protectedts/ptreconcile.(*Reconciler).run(0xc0011281e0, 0x688b290, 0xc0015e5b60, 0xc000a5c5a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/protectedts/ptreconcile/reconciler.go:121 +0x296
github.com/cockroachdb/cockroach/pkg/kv/kvserver/protectedts/ptreconcile.(*Reconciler).Start.func1(0x688b290, 0xc0015e5b60)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/protectedts/ptreconcile/reconciler.go:100 +0x47
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e5b60, 0xc000c30b80, 0x0, 0xc000e3a0f0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 213 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/server.(*Server).startSystemLogsGC.func1(0x688b290, 0xc0015e5bc0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server_systemlog_gc.go:177 +0x545
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0015e5bc0, 0xc000c30b80, 0x0, 0xc000e3a108)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 392 [select]:
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).Start.func9(0x688b290, 0xc000c240c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:895 +0x215
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002153b30, 0xc000c30b80, 0x0, 0xc002153b00)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 407 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).outputLoop(0xc00143e9a0, 0x688b1e8, 0xc000ef06c0, 0xc000ef06c0, 0xc0009fa8a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:267 +0x1b8
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).runOutputLoop(0xc00143e9a0, 0x688b1e8, 0xc000ef06c0, 0x6)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:289 +0xd4
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run.func1(0x688b1e8, 0xc000ef0540)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:275 +0x6d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b1e8, 0xc000ef0540, 0xc000c30b80, 0x0, 0xc000f515a8)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 3074 [semacquire, 4 minutes]:
sync.runtime_Semacquire(0xc002ae4040)
	/usr/local/go/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc002ae4038)
	/usr/local/go/src/sync/waitgroup.go:130 +0x65
golang.org/x/sync/errgroup.(*Group).Wait(0xc002ae4030, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:40 +0x31
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait(0xc002ae4030, 0x688b1e8, 0xc002586fc0, 0xc002b57140, 0xc002586fc0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:142 +0x57
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed.(*kvFeed).runUntilTableEvent(0xc0012fbba0, 0x688b1e8, 0xc00238adc0, 0x16b6985d65f3cd0b, 0x16b6980000000000, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed/kv_feed.go:374 +0x3f1
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed.(*kvFeed).run(0xc0012fbba0, 0x688b1e8, 0xc00238adc0, 0x1449d83, 0x688b290)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed/kv_feed.go:213 +0x12a
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc0027276e0, 0xc002727740)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 499 [semacquire, 4 minutes]:
sync.runtime_Semacquire(0xc0020bb990)
	/usr/local/go/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc0020bb988)
	/usr/local/go/src/sync/waitgroup.go:130 +0x65
golang.org/x/sync/errgroup.(*Group).Wait(0xc0020bb980, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:40 +0x31
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait(0xc0020bb980, 0x688b1e8, 0xc0013aabc0, 0xc0020cfda0, 0xc0013aabc0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:142 +0x57
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed(0xc00080c000, 0x688b290, 0xc001f801e0, 0xc001f68d20, 0x1, 0x8, 0xc001f68d28, 0x1, 0x1, 0x16b6985d172a5554, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:93 +0x4f8
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*dbAdapter).RangeFeed(0xc000d30b70, 0x688b290, 0xc001f801e0, 0xc001f68d20, 0x1, 0x8, 0xc001f68d28, 0x1, 0x1, 0x16b6985d172a5554, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/db_adapter.go:69 +0xb4
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).run.func1(0x688b290, 0xc001f801e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:251 +0xe6
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001f801e0, 0xc000c30b80, 0x0, 0xc0020bb920)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 1177 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).outputLoop(0xc00210d340, 0x688b1e8, 0xc002081ec0, 0xc002081ec0, 0xc001f53f10)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:267 +0x1b8
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).runOutputLoop(0xc00210d340, 0x688b1e8, 0xc002081ec0, 0x10)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:289 +0xd4
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run.func1(0x688b1e8, 0xc002081dc0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:275 +0x6d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b1e8, 0xc002081dc0, 0xc000c30b80, 0x0, 0xc002095db8)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 1221 [IO wait, 4 minutes]:
internal/poll.runtime_pollWait(0x7f32aa3eade0, 0x72, 0x0)
	/usr/local/go/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc0006fc598, 0x72, 0x0, 0x0, 0x50c4331)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc0006fc580, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:401 +0x212
net.(*netFD).accept(0xc0006fc580, 0x30, 0x7f32d4934108, 0x10000000030)
	/usr/local/go/src/net/fd_unix.go:172 +0x45
net.(*TCPListener).accept(0xc00198a2b8, 0x43a0b8, 0x30, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:139 +0x32
net.(*TCPListener).Accept(0xc00198a2b8, 0x688b290, 0xc002dee5a0, 0x50d47b0, 0xc)
	/usr/local/go/src/net/tcpsock.go:261 +0x65
github.com/cockroachdb/cockroach/pkg/util/netutil.(*Server).ServeWith(0xc002c4e198, 0x688b290, 0xc002dee5a0, 0xc000a5c5a0, 0x68660f0, 0xc00198a2b8, 0xc002e0ae10, 0x16b6985d257ddc14, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:138 +0x50
github.com/cockroachdb/cockroach/pkg/server.(*SQLServer).startServeSQL.func1(0x688b290, 0xc002dee5a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2381 +0xdc
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002dee5a0, 0xc000c30b80, 0x0, 0xc002de8f00)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 2218 [runnable]:
github.com/cockroachdb/redact/internal/rfmt.(*pp).printArg(0xc0002502d0, 0x4cc55e0, 0xc000a05110, 0x76)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/redact/internal/rfmt/print.go:698 +0x11d8
github.com/cockroachdb/redact/internal/rfmt.(*pp).doPrint(0xc0002502d0, 0xc003ab2600, 0x1, 0x1)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/redact/internal/rfmt/print.go:1276 +0x105
github.com/cockroachdb/redact/internal/rfmt.Sprint(0xc003ab2600, 0x1, 0x1, 0xc000a05110, 0xc002d73000)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/redact/internal/rfmt/print.go:257 +0x52
github.com/cockroachdb/redact.Sprint(...)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/redact/markers_print.go:30
github.com/cockroachdb/errors/errutil.NewWithDepth(0x1, 0x51057ce, 0x18, 0x78, 0x35)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/errors/errutil/utilities.go:44 +0xbd
github.com/cockroachdb/errors.New(...)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/errors/errutil_api.go:36
github.com/cockroachdb/pebble/sstable.(*Writer).Close(0xc002d74000, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/sstable/writer.go:808 +0xd29
github.com/cockroachdb/cockroach/pkg/storage.(*SSTWriter).Close(0xc003ab2e68)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/sst_writer.go:300 +0x33
github.com/cockroachdb/cockroach/pkg/storage.pebbleExportToSst(0x688b290, 0xc001d0ab70, 0x68fdfe0, 0xc002d91600, 0xc003a6ffa0, 0x5, 0x8, 0x0, 0x0, 0xc003a6ffa8, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/pebble.go:2140 +0x1149
github.com/cockroachdb/cockroach/pkg/storage.(*pebbleReadOnly).ExportMVCCToSst(0xc0015d3200, 0x688b290, 0xc001d0ab70, 0xc003a6ffa0, 0x5, 0x8, 0x0, 0x0, 0xc003a6ffa8, 0x5, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/pebble.go:1611 +0x138
github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval.evalExport(0x688b290, 0xc001d0ab70, 0x68fe0b0, 0xc0015d3200, 0x69da540, 0xc0019ee000, 0x16b6985d40d35b14, 0x0, 0x0, 0x100000001, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval/cmd_export.go:170 +0x874
github.com/cockroachdb/cockroach/pkg/kv/kvserver.evaluateCommand(0x688b290, 0xc001d0ab70, 0x69d90e8, 0xc0015d3200, 0x69da540, 0xc0019ee000, 0x0, 0x16b6985d40d35b14, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_evaluate.go:493 +0x715
github.com/cockroachdb/cockroach/pkg/kv/kvserver.evaluateBatch(0x688b290, 0xc001d0ab70, 0x0, 0x0, 0x69d90e8, 0xc0015d3200, 0x69da540, 0xc0019ee000, 0x0, 0xc005f837a0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_evaluate.go:268 +0x3d9
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).executeReadOnlyBatchWithServersideRefreshes(0xc0019ee000, 0x688b290, 0xc001d0ab70, 0x69d90e8, 0xc0015d3200, 0x69da540, 0xc0019ee000, 0xc005f837a0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_read.go:291 +0x28d
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).executeReadOnlyBatch(0xc0019ee000, 0x688b290, 0xc001d0ab70, 0xc005f837a0, 0xc0020ef040, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_read.go:85 +0x390
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).executeBatchWithConcurrencyRetries(0xc0019ee000, 0x688b290, 0xc001d0ab70, 0xc005f837a0, 0x5771998, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_send.go:429 +0x387
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).sendWithRangeID(0xc0019ee000, 0x688b290, 0xc001d0ab40, 0x6, 0xc005f837a0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_send.go:155 +0x2d2
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).Send(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_send.go:95
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).Send(0xc00026c800, 0x688b290, 0xc001d0ab10, 0x16b6985d40d35b14, 0x0, 0x0, 0x100000001, 0x1, 0x0, 0x6, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store_send.go:190 +0x62c
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).Send(0xc0002d02a0, 0x688b290, 0xc001d0ab10, 0x16b6985d40d35b14, 0x0, 0x0, 0x100000001, 0x1, 0x0, 0x6, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/stores.go:191 +0x11e
github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal.func1(0x688b290, 0xc001d0ab10, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:928 +0x1f8
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr(0xc000a5c5a0, 0x688b290, 0xc001d0ab10, 0x50e340d, 0x10, 0xc003ab7c38, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:347 +0xb2
github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal(0xc000238a00, 0x688b290, 0xc001d0ab10, 0x1, 0xc005f836c0, 0x4c06300, 0xc00198b560, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:916 +0x192
github.com/cockroachdb/cockroach/pkg/server.(*Node).Batch(0xc000238a00, 0x688b290, 0xc001d0aae0, 0xc005f836c0, 0xc0026a5c80, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:972 +0x1a8
github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.Batch(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:482
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*grpcTransport).sendBatch(0xc004eccc80, 0x688b290, 0xc001d0aae0, 0x1, 0x68f4a38, 0xc0019865f0, 0x16b6985d40d35b14, 0x0, 0x0, 0x100000001, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/transport.go:203 +0x176
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*grpcTransport).SendNext(0xc004eccc80, 0x688b1e8, 0xc0026a5c80, 0x16b6985d40d35b14, 0x0, 0x0, 0x100000001, 0x1, 0x0, 0x6, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/transport.go:185 +0x198
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).sendToReplicas(0xc00080c000, 0x688b1e8, 0xc0026a5c80, 0x16b6985d40d35b14, 0x0, 0x0, 0x0, 0x0, 0x0, 0x6, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender.go:1966 +0xb17
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).sendPartialBatch(0xc00080c000, 0x688b1e8, 0xc0026a5c80, 0x16b6985d40d35b14, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender.go:1558 +0x35f
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).divideAndSendBatchToRanges(0xc00080c000, 0x688b1e8, 0xc0026a5c80, 0x16b6985d40d35b14, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender.go:1170 +0x388
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).Send(0xc00080c000, 0x688b1e8, 0xc0026a5c80, 0x16b6985d40d35b14, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender.go:804 +0x61b
github.com/cockroachdb/cockroach/pkg/kv.(*CrossRangeTxnWrapperSender).Send(0xc000c01258, 0x688b1e8, 0xc0026a5c80, 0x16b6985d40d35b14, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/db.go:221 +0xb5
github.com/cockroachdb/cockroach/pkg/kv.SendWrappedWithAdmission(0x688b1e8, 0xc0026a5c80, 0x67ed0e0, 0xc000c01258, 0x16b6985d40d35b14, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/sender.go:451 +0x175
github.com/cockroachdb/cockroach/pkg/kv.SendWrappedWith(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/sender.go:432
github.com/cockroachdb/cockroach/pkg/sql/catalog/lease.getDescriptorsFromStoreForInterval(0x688b1e8, 0xc0026a5c80, 0xc000c01200, 0xc000d30d08, 0xc000d30d08, 0x0, 0x35, 0x16b6985d65f3cd0b, 0x1, 0x16b6985d40d35b14, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/lease/lease.go:233 +0x365
github.com/cockroachdb/cockroach/pkg/sql/catalog/lease.(*Manager).readOlderVersionForTimestamp(0xc000d392c0, 0x688b1e8, 0xc0026a5c80, 0xc000000035, 0x16b6985d65f3cd0b, 0x1, 0x0, 0x67f3600, 0xc00074ed50, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/lease/lease.go:325 +0x113
github.com/cockroachdb/cockroach/pkg/sql/catalog/lease.(*Manager).Acquire(0xc000d392c0, 0x688b1e8, 0xc0026a5c80, 0x16b6985d65f3cd0b, 0x1, 0x35, 0xc0015a05a0, 0x67ee5a0, 0xc0015a05a0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/lease/lease.go:922 +0x258
github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*leasedDescriptors).getByID(0xc00209e8d8, 0x688b1e8, 0xc0026a5c80, 0x683fd30, 0xc0029318c0, 0x35, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/leased_descriptors.go:136 +0x39c
github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).getDescriptorByIDMaybeSetTxnDeadline.func1(0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:117 +0x2fc
github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).getDescriptorByIDMaybeSetTxnDeadline(0xc00209e8c0, 0x688b1e8, 0xc0026a5c80, 0xc0029318c0, 0x35, 0xc003000000, 0x1ab45ed, 0x0, 0xc003460be0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:131 +0xcd
github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).getDescriptorByID(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:70
github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).getTypeByID(0xc00209e8c0, 0x688b1e8, 0xc0026a5c80, 0xc0029318c0, 0x35, 0x0, 0x0, 0x0, 0x4, 0xc003460cc8, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/type.go:114 +0xa5
github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).GetImmutableTypeByID(0xc00209e8c0, 0x688b1e8, 0xc0026a5c80, 0xc0029318c0, 0xc000000035, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/type.go:108 +0x85
github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).hydrateTypesInTableDesc.func2(0x688b1e8, 0xc0026a5c80, 0xc000000035, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/hydrate.go:82 +0xb2
github.com/cockroachdb/cockroach/pkg/sql/catalog/typedesc.TypeLookupFunc.GetTypeDescriptor(0xc002eb05b8, 0x688b1e8, 0xc0026a5c80, 0xc000000035, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/typedesc/type_desc.go:713 +0xa5
github.com/cockroachdb/cockroach/pkg/sql/catalog/hydratedtables.canUseCachedDescriptor(0x688b1e8, 0xc0026a5c80, 0xc0026cc030, 0x67f83e0, 0xc002eb05b8, 0x688b1e8, 0xc0026a5c80, 0x683fd30)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/hydratedtables/hydratedcache.go:269 +0x105
github.com/cockroachdb/cockroach/pkg/sql/catalog/hydratedtables.(*Cache).GetHydratedTableDescriptor(0xc0004d3900, 0x688b1e8, 0xc0026a5c80, 0x69f1d10, 0xc002668500, 0x67f83e0, 0xc002eb05b8, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/hydratedtables/hydratedcache.go:195 +0x4a5
github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).hydrateTypesInTableDesc(0xc00209e8c0, 0x688b1e8, 0xc0026a5c80, 0xc0029318c0, 0x69f1d10, 0xc002668500, 0x7f32aa36f660, 0xc002668500, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/hydrate.go:102 +0x4ee
github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).getTableByID(0xc00209e8c0, 0x688b1e8, 0xc0026a5c80, 0xc0029318c0, 0x37, 0x0, 0x0, 0x0, 0xc002668e20, 0xc002668df0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/table.go:141 +0x1f1
github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).GetImmutableTableByID(0xc00209e8c0, 0x688b1e8, 0xc0026a5c80, 0xc0029318c0, 0x37, 0x0, 0x0, 0x0, 0xc002668c60, 0x68f1af8, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/table.go:118 +0x85
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*rowFetcherCache).TableDescForKey.func1(0x688b1e8, 0xc0026a5c80, 0xc0029318c0, 0xc002676240, 0x3ff0000000000000)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/rowfetcher_cache.go:108 +0xe5
github.com/cockroachdb/cockroach/pkg/kv.(*DB).Txn.func1(0x688b1e8, 0xc0026a5c80, 0xc0029318c0, 0xc002676240, 0x3ff0000000000000)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/db.go:824 +0x43
github.com/cockroachdb/cockroach/pkg/kv.(*Txn).exec(0xc0029318c0, 0x688b1e8, 0xc0026a5c80, 0xc003abb700, 0xc0029318c0, 0x1ebf620)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/txn.go:943 +0xdc
github.com/cockroachdb/cockroach/pkg/kv.(*DB).Txn(0xc000c01200, 0x688b1e8, 0xc0026a5c80, 0xc0034617d8, 0xc002668500, 0x37)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/db.go:823 +0x105
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*rowFetcherCache).TableDescForKey(0xc001d40580, 0x688b1e8, 0xc0026a5c80, 0xc0019fa328, 0x8, 0x18, 0x16b6985d65f3cd0b, 0x4046800000000001, 0x36ec3701, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/rowfetcher_cache.go:103 +0x49b
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*kvEventToRowConsumer).eventToRow(0xc0012fb110, 0x688b1e8, 0xc0026a5c80, 0xc0019fa328, 0x8, 0x18, 0xc0019fa33a, 0x5, 0x6, 0x16b6985d62ba9663, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_processors.go:730 +0x205
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*kvEventToRowConsumer).ConsumeEvent(0xc0012fb110, 0x688b1e8, 0xc0026a5c80, 0xc0019fa328, 0x8, 0x18, 0xc0019fa33a, 0x5, 0x6, 0x16b6985d62ba9663, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_processors.go:672 +0x19e
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changeAggregator).tick(0xc0024fb100, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_processors.go:513 +0x3b5
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changeAggregator).Next(0xc0024fb100, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_processors.go:468 +0x85
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changeFrontier).Next(0xc0026ac000, 0xc000341420, 0x65, 0x68500d0, 0xc000126840)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_processors.go:1230 +0xec
github.com/cockroachdb/cockroach/pkg/sql/execinfra.Run(0x688b1e8, 0xc0026a5b40, 0x68aba78, 0xc0026ac000, 0x6840f28, 0xc002836a80)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/execinfra/base.go:186 +0x35
github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBaseNoHelper).Run(0xc0026ac000, 0x688b1e8, 0xc0026a5b40)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/execinfra/processorsbase.go:732 +0x90
github.com/cockroachdb/cockroach/pkg/sql/flowinfra.(*FlowBase).Run(0xc0013a3800, 0x688b1e8, 0xc0026a5b40, 0x5772800)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/flowinfra/flow.go:438 +0x2d8
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run(0xc0004de300, 0xc000e75680, 0x0, 0xc002e2b0b0, 0xc001a1f880, 0xc002836380, 0xc002e2af50, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:538 +0x64f
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeeddist.StartDistChangefeed(0x688b290, 0xc0020f8d20, 0x68fd210, 0xc0022d7ff8, 0x9d9785f204a0001, 0xc002534060, 0xc0022208d0, 0x18, 0xc002534030, 0x16b6985d4a8ab183, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeeddist/distflow.go:144 +0xb0e
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.distChangefeedFlow(0x688b290, 0xc0020f8d20, 0x68fd210, 0xc0022d7ff8, 0x9d9785f204a0001, 0xc002534060, 0xc0022208d0, 0x18, 0xc002534030, 0x16b6985d4a8ab183, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_dist.go:117 +0x387
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changefeedResumer).resumeWithRetries(0xc0022d7a68, 0x688b290, 0xc0020f8d20, 0x68fd210, 0xc0022d7ff8, 0x9d9785f204a0001, 0xc002534060, 0xc0022208d0, 0x18, 0xc002534030, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_stmt.go:716 +0x215
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changefeedResumer).Resume(0xc0022d7a68, 0x688b290, 0xc0020f8d20, 0x4de6ba0, 0xc0022d7ff8, 0xc000ed0160, 0x100)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_stmt.go:648 +0x247
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*reportErrorResumer).Resume(0xc00276fb78, 0x688b290, 0xc0020f8d20, 0x4de6ba0, 0xc0022d7ff8, 0x4966200, 0xc001373788)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/testfeed_test.go:209 +0x5b
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine.func2(0xc0003aa140, 0x683f970, 0xc00276fb78, 0x688b290, 0xc0020f8d20, 0x4de6ba0, 0xc0022d7ff8, 0xc003abd958)
	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:1202 +0xae
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine(0xc000804fc0, 0x688b290, 0xc0020f8c90, 0x4de6ba0, 0xc0022d7ff8, 0x683f970, 0xc00276fb78, 0xc000ed0160, 0x50c569b, 0x7, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:1203 +0x987
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).runJob(0xc000804fc0, 0x688b1e8, 0xc002392ac0, 0x683f970, 0xc00276fb78, 0xc000ed0160, 0x50c569b, 0x7, 0xc002220798, 0x16, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/adopt.go:414 +0x4c9
github.com/cockroachdb/cockroach/pkg/jobs.(*StartableJob).Start.func2(0x68a71b0, 0xc00228d4d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/jobs.go:915 +0xf6
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x68a71b0, 0xc00228d4d0, 0xc00228d340, 0x0, 0xc00228a040)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 3041 [select]:
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/schemafeed.(*schemaFeed).pollTableHistory(0xc0012fb040, 0x688b1e8, 0xc00238adc0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/schemafeed/schema_feed.go:319 +0x155
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/schemafeed.(*schemaFeed).Run(0xc0012fb040, 0x688b1e8, 0xc00238adc0, 0x1449d83, 0x688b1e8)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/schemafeed/schema_feed.go:265 +0xb0
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc0027276e0, 0xc002727710)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 3243 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).rangeFeedWithRangeID(0xc001514000, 0x2d, 0xc001e71200, 0x68e13e8, 0xc002553fc0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_rangefeed.go:244 +0x670
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).RangeFeed(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_rangefeed.go:142
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).RangeFeed(0xc00026c800, 0xc001e71200, 0x68e13e8, 0xc002553fc0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:2661 +0x136
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).RangeFeed(0xc0002d02a0, 0xc001e71200, 0x68e13e8, 0xc002553fc0, 0xc001696f68)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/stores.go:216 +0xce
github.com/cockroachdb/cockroach/pkg/server.(*Node).RangeFeed(0xc000238a00, 0xc001e71200, 0x68e13e8, 0xc002553fc0, 0xc001696f68, 0x1ab3fba)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:1090 +0x54
github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.RangeFeed.func1(0xc002c2ed20, 0x68f1c78, 0xc000238a00, 0xc001e71200, 0x688b1e8, 0xc0025313c0, 0xc002897c20, 0xc002897c80)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:617 +0xc5
created by github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.RangeFeed
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:615 +0x12e

goroutine 1169 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvprober.(*Prober).Start.func1.1(0x688b290, 0xc0020816c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvprober/kvprober.go:188 +0x3dd
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002327b00, 0xc000c30b80, 0x0, 0xc0016a4d20)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 3040 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changeFrontier).Start.func1(0x688b1e8, 0xc0026a5b40, 0xc0026ac000)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_processors.go:1163 +0x48
created by github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changeFrontier).Start
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_processors.go:1162 +0x43b

goroutine 1168 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvprober.(*Prober).Start.func1.1(0x688b290, 0xc002081500)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvprober/kvprober.go:188 +0x3dd
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002327ad0, 0xc000c30b80, 0x0, 0xc0016a4cd0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 2271 [select]:
google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc002ef8fa0, 0x1, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:406 +0xff
google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc003018360, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:533 +0x1dd
google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc002d8b860)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:405 +0x7b
created by google.golang.org/grpc/internal/transport.newHTTP2Client
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:403 +0x12ae

goroutine 405 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run(0xc000bea000, 0x688b290, 0xc0021911d0, 0x6, 0xc0021911a0, 0xc000a5c5a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:249 +0x3f6
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).Start.func1(0x688b290, 0xc0021911d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:204 +0x59
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0021911d0, 0xc000c30b80, 0x0, 0xc0007f3c80)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 390 [select]:
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).Start.func7(0x688b290, 0xc002153aa0)
	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:840 +0x1ef
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002153aa0, 0xc000c30b80, 0x0, 0xc000f685c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 404 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).rangeFeedWithRangeID(0xc0019ee000, 0x6, 0xc001416b00, 0x68e13e8, 0xc0007f3c00, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_rangefeed.go:244 +0x670
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).RangeFeed(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_rangefeed.go:142
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).RangeFeed(0xc00026c800, 0xc001416b00, 0x68e13e8, 0xc0007f3c00, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:2661 +0x136
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).RangeFeed(0xc0002d02a0, 0xc001416b00, 0x68e13e8, 0xc0007f3c00, 0xc001659768)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/stores.go:216 +0xce
github.com/cockroachdb/cockroach/pkg/server.(*Node).RangeFeed(0xc000238a00, 0xc001416b00, 0x68e13e8, 0xc0007f3c00, 0xc001659768, 0x1ab3fba)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:1090 +0x54
github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.RangeFeed.func1(0xc0009fa6f0, 0x68f1c78, 0xc000238a00, 0xc001416b00, 0x688b1e8, 0xc000ef0500, 0xc002122840, 0xc0021228a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:617 +0xc5
created by github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.RangeFeed
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:615 +0x12e

goroutine 3088 [select]:
github.com/cockroachdb/cockroach/pkg/rpc.respStreamClientAdapter.recvInternal(0x688b1e8, 0xc002531200, 0xc002897800, 0xc002897860, 0xc00211d8c8, 0x2, 0x2, 0xc00211d718)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:560 +0xb4
github.com/cockroachdb/cockroach/pkg/rpc.rangeFeedClientAdapter.Recv(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:589
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).singleRangeFeed(0xc00080c000, 0x688b1e8, 0xc002587040, 0xc001fd4840, 0x2, 0x8, 0xc001972998, 0x6, 0x8, 0x16b6985d65f3cd0b, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:401 +0x959
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).partialRangeFeed(0xc00080c000, 0x688b1e8, 0xc002587040, 0xc00143c1b0, 0xc001fd4840, 0x2, 0x8, 0xc001972998, 0x6, 0x8, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:277 +0x4a9
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed.func1.1(0x688b1e8, 0xc002587040, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:80 +0x145
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002ae41b0, 0xc0027f2ab0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 388 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql/catalog/lease.(*Manager).RefreshLeases.func1(0x688b290, 0xc0021539b0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/lease/lease.go:996 +0xf8
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0021539b0, 0xc000c30b80, 0x0, 0xc002153980)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 403 [select]:
github.com/cockroachdb/cockroach/pkg/rpc.respStreamClientAdapter.recvInternal(0x688b1e8, 0xc000ef0500, 0xc002122840, 0xc0021228a0, 0xc001a5b8c8, 0x2, 0x2, 0xc001a5b718)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:560 +0xb4
github.com/cockroachdb/cockroach/pkg/rpc.rangeFeedClientAdapter.Recv(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:589
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).singleRangeFeed(0xc00080c000, 0x688b1e8, 0xc000c19d00, 0xc0014993b0, 0x1, 0x8, 0xc0014993b8, 0x1, 0x1, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:401 +0x959
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).partialRangeFeed(0xc00080c000, 0x688b1e8, 0xc000c19d00, 0xc001a41290, 0xc0014993b0, 0x1, 0x8, 0xc0014993b8, 0x1, 0x1, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:277 +0x4a9
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed.func1.1(0x688b1e8, 0xc000c19d00, 0xc000a5c5a0, 0x5776f70)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:80 +0x145
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc00156bec0, 0xc002190ff0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 389 [select, 2 minutes]:
github.com/cockroachdb/cockroach/pkg/sql/catalog/lease.(*Manager).PeriodicallyRefreshSomeLeases.func1(0x688b290, 0xc0021539e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/lease/lease.go:1105 +0x2ec
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0021539e0, 0xc000c30b80, 0x0, 0xc0009ee380)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 386 [select]:
github.com/cockroachdb/cockroach/pkg/sql/stmtdiagnostics.(*Registry).poll(0xc0006dca20, 0x688b290, 0xc002153860)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/stmtdiagnostics/statement_diagnostics.go:147 +0x2ba
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002153860, 0xc000c30b80, 0x0, 0xc0009ee2a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 387 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).processEvents(0xc00170f600, 0x688b290, 0xc002153950, 0xc002153890, 0xc00150eea0, 0xc00150ef00, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:347 +0xed
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).run(0xc00170f600, 0x688b290, 0xc002153950, 0xc002153890)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:257 +0x605
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).Start.func1(0x688b290, 0xc002153950)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:184 +0x47
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002153950, 0xc000c30b80, 0x0, 0xc0013de8d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 319 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed.func1(0x688b1e8, 0xc000c19d00, 0xc000a5c5a0, 0x5776f70)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:76 +0x145
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc00156bec0, 0xc00156bef0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 318 [semacquire, 4 minutes]:
sync.runtime_Semacquire(0xc00156bed0)
	/usr/local/go/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc00156bec8)
	/usr/local/go/src/sync/waitgroup.go:130 +0x65
golang.org/x/sync/errgroup.(*Group).Wait(0xc00156bec0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:40 +0x31
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait(0xc00156bec0, 0x688b1e8, 0xc000c19d00, 0xc002122180, 0xc000c19d00)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:142 +0x57
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed(0xc00080c000, 0x688b290, 0xc002153950, 0xc0014993b0, 0x1, 0x8, 0xc0014993b8, 0x1, 0x1, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:93 +0x4f8
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*dbAdapter).RangeFeed(0xc000d30b70, 0x688b290, 0xc002153950, 0xc0014993b0, 0x1, 0x8, 0xc0014993b8, 0x1, 0x1, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/db_adapter.go:69 +0xb4
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).run.func1(0x688b290, 0xc002153950)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:251 +0xe6
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002153950, 0xc000c30b80, 0x0, 0xc00156be60)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 129 [select]:
github.com/cockroachdb/cockroach/pkg/sql/stats.(*Refresher).Start.func1(0x688b290, 0xc002153830)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/stats/automatic_stats.go:262 +0x18a
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002153830, 0xc000c30b80, 0x0, 0xc000f68480)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 1222 [select, 4 minutes]:
database/sql.(*DB).connectionOpener(0xc000f5a340, 0x688b1e8, 0xc002de8080)
	/usr/local/go/src/database/sql/sql.go:1133 +0xb5
created by database/sql.OpenDB
	/usr/local/go/src/database/sql/sql.go:740 +0x12a

goroutine 215 [select]:
github.com/cockroachdb/cockroach/pkg/sql/sqlliveness/slstorage.(*Storage).deleteSessionsLoop(0xc0003eaa50, 0x688b290, 0xc000c0b900)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlliveness/slstorage/slstorage.go:319 +0x1fa
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001f80030, 0xc000c30b80, 0x0, 0xc0009c6c70)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 216 [select]:
github.com/cockroachdb/cockroach/pkg/sql/sqlliveness/slinstance.(*Instance).heartbeatLoop(0xc001128a20, 0x688b1e8, 0xc000b85c80)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlliveness/slinstance/slinstance.go:227 +0x178
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001f800f0, 0xc000c30b80, 0x0, 0xc0009c6cc0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 217 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).processEvents(0xc0014fe000, 0x688b290, 0xc001f801e0, 0xc001f80120, 0xc001874480, 0xc0018744e0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:347 +0xed
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).run(0xc0014fe000, 0x688b290, 0xc001f801e0, 0xc001f80120)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:257 +0x605
github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).Start.func1(0x688b290, 0xc001f801e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:184 +0x47
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001f801e0, 0xc000c30b80, 0x0, 0xc000e3a318)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 224 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql/gcjob/gcjobnotifier.(*Notifier).run(0xc001a460c0, 0x688b290, 0xc001f81020)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/gcjob/gcjobnotifier/notifier.go:136 +0xea
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001f81020, 0xc000c30b80, 0x0, 0xc0009c72b0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 2138 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*teeGroup).tee.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/testfeed_test.go:982 +0x1b8
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002a36fc0, 0xc002222300)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 127 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql/sqlstats/persistedsqlstats.(*jobMonitor).start.func1(0x688b290, 0xc0021537d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlstats/persistedsqlstats/scheduled_job_monitor.go:76 +0x125
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0021537d0, 0xc000c30b80, 0x0, 0xc0013de828)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 225 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*TemporaryObjectCleaner).Start.func1(0x688b290, 0xc001f81050)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/temporary_schema.go:629 +0x2de
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001f81050, 0xc000c30b80, 0x0, 0xc000e3a798)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 370 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql/flowinfra.(*FlowScheduler).Start.func1(0x688b290, 0xc001f81080)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/flowinfra/flow_scheduler.go:318 +0x27e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001f81080, 0xc000c30b80, 0x0, 0xc00079f000)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 371 [select]:
github.com/cockroachdb/cockroach/pkg/sql/idxusage.(*LocalIndexUsageStats).startStatsIngestionLoop.func1(0x688b290, 0xc001f810b0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/idxusage/local_idx_usage_stats.go:426 +0x10b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001f810b0, 0xc000c30b80, 0x0, 0xc000e3a7b0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 372 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql/sqlstats/persistedsqlstats.(*PersistedSQLStats).startSQLStatsFlushLoop.func1(0x688b290, 0xc001f810e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlstats/persistedsqlstats/provider.go:142 +0x293
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc001f810e0, 0xc000c30b80, 0x0, 0xc000e3a7c8)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 503 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).rangeFeedWithRangeID(0xc0015d0000, 0x2a, 0xc0020ae300, 0x68e13e8, 0xc000ecfe60, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_rangefeed.go:244 +0x670
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).RangeFeed(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_rangefeed.go:142
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).RangeFeed(0xc00026c800, 0xc0020ae300, 0x68e13e8, 0xc000ecfe60, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:2661 +0x136
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).RangeFeed(0xc0002d02a0, 0xc0020ae300, 0x68e13e8, 0xc000ecfe60, 0xc0016aaf68)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/stores.go:216 +0xce
github.com/cockroachdb/cockroach/pkg/server.(*Node).RangeFeed(0xc000238a00, 0xc0020ae300, 0x68e13e8, 0xc000ecfe60, 0xc0016aaf68, 0x1ab3fba)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:1090 +0x54
github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.RangeFeed.func1(0xc0020f00f0, 0x68f1c78, 0xc000238a00, 0xc0020ae300, 0x688b1e8, 0xc0013aac00, 0xc0020cfe60, 0xc0020cfec0)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:617 +0xc5
created by github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.RangeFeed
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:615 +0x12e

goroutine 128 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql/sqlstats/sslocal.(*SQLStats).Start.func1(0x688b290, 0xc002153800)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlstats/sslocal/sslocal_provider.go:80 +0x312
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002153800, 0xc000c30b80, 0x0, 0xc0013de840)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 3225 [semacquire, 4 minutes]:
sync.runtime_Semacquire(0xc002ae4100)
	/usr/local/go/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc002ae40f8)
	/usr/local/go/src/sync/waitgroup.go:130 +0x65
golang.org/x/sync/errgroup.(*Group).Wait(0xc002ae40f0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:40 +0x31
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait(0xc002ae40f0, 0x688b1e8, 0xc002587000, 0xc002a57500, 0xc002587000)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:142 +0x57
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed.rangefeedFactory.Run(0xc001de5e70, 0x688b1e8, 0xc002586fc0, 0x7f32a97de838, 0xc002aeef50, 0xc002726180, 0x1, 0x1, 0x16b6985d65f3cd0b, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed/physical_kv_feed.go:80 +0x355
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed.(*kvFeed).runUntilTableEvent.func3(0x688b1e8, 0xc002586fc0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed/kv_feed.go:367 +0x123
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002ae4030, 0xc002ae4090)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 3242 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).outputLoop(0xc00240b0a0, 0x688b1e8, 0xc002531340, 0xc002531340, 0xc002c2ecf0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:267 +0x1b8
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).runOutputLoop(0xc00240b0a0, 0x688b1e8, 0xc002531340, 0x2f)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:289 +0xd4
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run.func1(0x688b1e8, 0xc002531240)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:275 +0x6d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b1e8, 0xc002531240, 0xc000c30b80, 0x0, 0xc002a68138)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 3089 [select]:
github.com/cockroachdb/cockroach/pkg/rpc.respStreamClientAdapter.recvInternal(0x688b1e8, 0xc0025313c0, 0xc002897c20, 0xc002897c80, 0xc0021198c8, 0x2, 0x2, 0xc002119718)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:560 +0xb4
github.com/cockroachdb/cockroach/pkg/rpc.rangeFeedClientAdapter.Recv(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:589
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).singleRangeFeed(0xc00080c000, 0x688b1e8, 0xc002587040, 0xc001972998, 0x6, 0x8, 0xc0020b0368, 0x6, 0x8, 0x16b6985d65f3cd0b, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:401 +0x959
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).partialRangeFeed(0xc00080c000, 0x688b1e8, 0xc002587040, 0xc00143c1b0, 0xc001972998, 0x6, 0x8, 0xc0020b0368, 0x6, 0x8, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:277 +0x4a9
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed.func1.1(0x688b1e8, 0xc002587040, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:80 +0x145
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002ae41b0, 0xc0027f2ae0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 3236 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run(0xc001440420, 0x688b290, 0xc0027f2cf0, 0x2e, 0xc0027f2cc0, 0xc000a5c5a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:249 +0x3f6
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).Start.func1(0x688b290, 0xc0027f2cf0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:204 +0x59
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0027f2cf0, 0xc000c30b80, 0x0, 0xc002553dc0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 3036 [semacquire, 4 minutes]:
sync.runtime_Semacquire(0xc0027276f0)
	/usr/local/go/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc0027276e8)
	/usr/local/go/src/sync/waitgroup.go:130 +0x65
golang.org/x/sync/errgroup.(*Group).Wait(0xc0027276e0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:40 +0x31
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait(0xc0027276e0, 0x688b1e8, 0xc00238adc0, 0xc001de5ea0, 0xc00238adc0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:142 +0x57
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed.Run(0x688b1e8, 0xc0026a5c80, 0xc0006cc000, 0xc000c01200, 0xc000d30d08, 0xc000d30d08, 0x0, 0xc00065e8c0, 0x509aa00, 0xc000cd64e0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed/kv_feed.go:105 +0x630
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changeAggregator).startKVFeed.func1(0x688b1e8, 0xc0026a5c80)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_processors.go:306 +0xfe
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b1e8, 0xc0026a5c80, 0xc000c30b80, 0x0, 0xc00198ba40)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 502 [select]:
github.com/cockroachdb/cockroach/pkg/rpc.respStreamClientAdapter.recvInternal(0x688b1e8, 0xc0013aac00, 0xc0020cfe60, 0xc0020cfec0, 0xc001e858c8, 0x2, 0x2, 0xc001e85718)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:560 +0xb4
github.com/cockroachdb/cockroach/pkg/rpc.rangeFeedClientAdapter.Recv(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:589
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).singleRangeFeed(0xc00080c000, 0x688b1e8, 0xc0013aabc0, 0xc001f68d20, 0x1, 0x8, 0xc001f68d28, 0x1, 0x1, 0x16b6985d172a5554, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:401 +0x959
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).partialRangeFeed(0xc00080c000, 0x688b1e8, 0xc0013aabc0, 0xc0006dad80, 0xc001f68d20, 0x1, 0x8, 0xc001f68d28, 0x1, 0x1, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:277 +0x4a9
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed.func1.1(0x688b1e8, 0xc0013aabc0, 0xc000a5c5a0, 0x5776f70)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:80 +0x145
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc002091b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc0020bb980, 0xc0020bbad0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 500 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed.func1(0x688b1e8, 0xc0013aabc0, 0xc000a5c5a0, 0x5776f70)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:76 +0x145
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc0020bb980, 0xc0020bb9b0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 504 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run(0xc000bcfce0, 0x688b290, 0xc0020bbcb0, 0x2a, 0xc0020bbc80, 0xc000a5c5a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:249 +0x3f6
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).Start.func1(0x688b290, 0xc0020bbcb0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:204 +0x59
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0020bbcb0, 0xc000c30b80, 0x0, 0xc000ecfee0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 506 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).outputLoop(0xc0010790a0, 0x688b1e8, 0xc0013aad40, 0xc0013aad40, 0xc0020f0170)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:267 +0x1b8
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).runOutputLoop(0xc0010790a0, 0x688b1e8, 0xc0013aad40, 0x2a)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:289 +0xd4
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run.func1(0x688b1e8, 0xc0013aac40)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:275 +0x6d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b1e8, 0xc0013aac40, 0xc000c30b80, 0x0, 0xc0016d2c48)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 3035 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*kafkaSink).workerLoop(0xc001f09d40)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/sink_kafka.go:338 +0x158
created by github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*kafkaSink).start
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/sink_kafka.go:164 +0x8b

goroutine 2290 [select]:
google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc002976000)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_server.go:1061 +0x265
created by google.golang.org/grpc/internal/transport.NewServerTransport
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_server.go:313 +0x11d5

goroutine 62985 [select]:
github.com/cockroachdb/cockroach/pkg/sql/stats.(*Refresher).Start.func1.1(0x688b290, 0xc002153830)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/stats/automatic_stats.go:274 +0xf9
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002153830, 0xc000c30b80, 0x0, 0xc002b3c8d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 1220 [select]:
github.com/cockroachdb/cockroach/pkg/ts.(*poller).start.func1(0x688b290, 0xc002dee4b0)
	/go/src/github.com/cockroachdb/cockroach/pkg/ts/db.go:168 +0xea
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002dee4b0, 0xc000c30b80, 0x0, 0xc002df25b0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 1838 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*kafkaFeed).Next(0xc002a37020, 0x6844a10, 0xc0032a3740, 0xc002dec000)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/testfeed_test.go:1199 +0xb5
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.readNextMessages(0x684eab0, 0xc002a37020, 0x2, 0x0, 0x51c0cec, 0x36, 0xc001a4db18, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/helpers_test.go:74 +0x82
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.assertPayloadsBaseErr(0x684eab0, 0xc002a37020, 0xc0032a3760, 0x2, 0x2, 0x0, 0x18, 0x4395bd5)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/helpers_test.go:166 +0x5d
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.assertPayloadsBase(0x69d05d0, 0xc00249f800, 0x684eab0, 0xc002a37020, 0xc0032a3760, 0x2, 0x2, 0xc0032a0000)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/helpers_test.go:160 +0x8c
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.assertPayloads(0x69d05d0, 0xc00249f800, 0x684eab0, 0xc002a37020, 0xc0032a3760, 0x2, 0x2)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/helpers_test.go:207 +0x89
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.TestChangefeedWorksOnRBRChange.func2.2(0xc00249f800)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_test.go:1907 +0x3ba
testing.tRunner(0xc00249f800, 0xc0025acce0)
	/usr/local/go/src/testing/testing.go:1193 +0xef
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1238 +0x2b3

goroutine 3224 [select]:
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvevent.(*blockingBuffer).Get(0xc002b57080, 0x688b1e8, 0xc002586fc0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvevent/blocking_buffer.go:137 +0xfc
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed.copyFromSourceToDestUntilTableEvent(0x688b1e8, 0xc002586fc0, 0x7f32a97de558, 0xc00198ba10, 0x7f32a97de890, 0xc002aeef50, 0xc002726180, 0x1, 0x1, 0x16b6985d65f3cd0b, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed/kv_feed.go:511 +0x357
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed.(*kvFeed).runUntilTableEvent.func2(0x688b1e8, 0xc002586fc0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed/kv_feed.go:364 +0x155
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002ae4030, 0xc002ae4060)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 2281 [chan receive, 4 minutes]:
google.golang.org/grpc.(*addrConn).resetTransport(0xc002465b80)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:1219 +0x465
created by google.golang.org/grpc.(*addrConn).connect
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:849 +0x12a

goroutine 3246 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).outputLoop(0xc00240b260, 0x688b1e8, 0xc002531500, 0xc002531500, 0xc002c2eda0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:267 +0x1b8
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).runOutputLoop(0xc00240b260, 0x688b1e8, 0xc002531500, 0x2d)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:289 +0xd4
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run.func1(0x688b1e8, 0xc002531400)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:275 +0x6d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b1e8, 0xc002531400, 0xc000c30b80, 0x0, 0xc002a68270)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 3244 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run(0xc0014406e0, 0x688b290, 0xc0027f3200, 0x2d, 0xc0027f31d0, 0xc000a5c5a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:249 +0x3f6
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).Start.func1(0x688b290, 0xc0027f3200)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:204 +0x59
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0027f3200, 0xc000c30b80, 0x0, 0xc002b14040)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 2172 [IO wait, 4 minutes]:
internal/poll.runtime_pollWait(0x7f32aa3eab28, 0x72, 0x0)
	/usr/local/go/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc002675218, 0x72, 0x0, 0x0, 0x50c4331)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc002675200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:401 +0x212
net.(*netFD).accept(0xc002675200, 0x15b13f1, 0xc001fe58cc, 0xc0028fc060)
	/usr/local/go/src/net/fd_unix.go:172 +0x45
net.(*TCPListener).accept(0xc00205b9b0, 0x0, 0xc002947e20, 0x68eff28)
	/usr/local/go/src/net/tcpsock_posix.go:139 +0x32
net.(*TCPListener).Accept(0xc00205b9b0, 0xc002947e70, 0x18, 0xc001922a80, 0x935abb)
	/usr/local/go/src/net/tcpsock.go:261 +0x65
net/http.(*Server).Serve(0xc002812000, 0x68660f0, 0xc00205b9b0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:2961 +0x285
net/http/httptest.(*Server).goServe.func1(0xc001fe5880)
	/usr/local/go/src/net/http/httptest/server.go:308 +0x6e
created by net/http/httptest.(*Server).goServe
	/usr/local/go/src/net/http/httptest/server.go:306 +0x5c

goroutine 2282 [select]:
github.com/cockroachdb/cockroach/pkg/rpc.(*Context).runHeartbeat(0xc000804d80, 0xc00245d0e0, 0xc001498580, 0xf, 0xc002178960, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:1203 +0x2a8
github.com/cockroachdb/cockroach/pkg/rpc.(*Context).grpcDialNodeInternal.func1.1(0x688b290, 0xc002b24ed0)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:1127 +0x7c
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002b24ed0, 0xc000c30b80, 0x0, 0xc00245d220)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 1165 [select]:
github.com/cockroachdb/cockroach/pkg/jobs.(*jobScheduler).runDaemon.func1(0x688b290, 0xc0023279b0)
	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/job_scheduler.go:365 +0x28f
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0023279b0, 0xc000c30b80, 0x0, 0xc00205bf98)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 3240 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run(0xc001440580, 0x688b290, 0xc0027f2f90, 0x2f, 0xc0027f2f60, 0xc000a5c5a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:249 +0x3f6
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).Start.func1(0x688b290, 0xc0027f2f90)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:204 +0x59
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0027f2f90, 0xc000c30b80, 0x0, 0xc002553f00)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 3203 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/sql/gcjob.schemaChangeGCResumer.Resume(0x9d9785fb82a8001, 0x688b290, 0xc003099ef0, 0x4de6ba0, 0xc003150820, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/gcjob/gc_job.go:115 +0x34c
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine.func2(0xc0003aa240, 0x6840ca8, 0xc0020b1e40, 0x688b290, 0xc003099ef0, 0x4de6ba0, 0xc003150820, 0xc002caf920)
	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:1202 +0xae
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine(0xc000804fc0, 0x688b290, 0xc003099e60, 0x4de6ba0, 0xc003150820, 0x6840ca8, 0xc0020b1e40, 0xc00185ec60, 0xc0020b1ba8, 0x7, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:1203 +0x987
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).runJob(0xc000804fc0, 0x688b1e8, 0xc002cc73c0, 0x6840ca8, 0xc0020b1e40, 0xc00185ec60, 0xc0020b1ba8, 0x7, 0xc00266f038, 0x16, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/adopt.go:414 +0x4c9
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).resumeJob.func1(0x688b1e8, 0xc000c240c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/adopt.go:336 +0x165
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b1e8, 0xc000c240c0, 0xc000c30b80, 0x0, 0xc0028a82d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 1166 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/closedts/sidetransport.(*Sender).Run.func2(0x688b290, 0xc002327a10)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/closedts/sidetransport/sender.go:250 +0x1a5
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc002327a10, 0xc000c30b80, 0x0, 0xc00205bfc8)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 1198 [sync.Cond.Wait, 4 minutes]:
sync.runtime_notifyListWait(0xc002eaef50, 0xc000000007)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc002eaef40)
	/usr/local/go/src/sync/cond.go:56 +0x99
github.com/cockroachdb/cockroach/pkg/sql.(*StmtBuf).CurCmd(0xc001f6f0c8, 0x0, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_io.go:441 +0x9e
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd(0xc002cfb300, 0x688b1e8, 0xc002eaef80, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1682 +0x65
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run(0xc002cfb300, 0x688b1e8, 0xc002eaef80, 0xc0004e0a00, 0x5400, 0x15000, 0xc0004e0c80, 0xc002ea6c20, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1658 +0x22f
github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn(0xc001a54700, 0x688b1e8, 0xc002eaef80, 0xc002cfb300, 0x5400, 0x15000, 0xc0004e0c80, 0xc002ea6c20, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:669 +0xce
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync.func1(0xc002e85bd0, 0xc0027579b0, 0x688b1e8, 0xc002eaef80, 0xc002ea6c20, 0xc001a54700, 0xc001f6ea80, 0x68eeea8, 0xc002ccd180, 0xc00243eb40, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:666 +0x3f1
created by github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:594 +0x191

goroutine 3228 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed.func1(0x688b1e8, 0xc002587040, 0xc000a5c5a0, 0x5776f70)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:76 +0x145
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002ae41b0, 0xc002ae41e0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 3034 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*teeGroup).tee.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/testfeed_test.go:982 +0x1b8
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002a36fc0, 0xc002726300)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 3234 [select]:
github.com/cockroachdb/cockroach/pkg/rpc.respStreamClientAdapter.recvInternal(0x688b1e8, 0xc002531000, 0xc0028972c0, 0xc0028973e0, 0xc0023438c8, 0x2, 0x2, 0xc002343718)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:560 +0xb4
github.com/cockroachdb/cockroach/pkg/rpc.rangeFeedClientAdapter.Recv(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:589
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).singleRangeFeed(0xc00080c000, 0x688b1e8, 0xc002587040, 0xc0020b0368, 0x6, 0x8, 0xc001fd4848, 0x2, 0x2, 0x16b6985d65f3cd0b, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:401 +0x959
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).partialRangeFeed(0xc00080c000, 0x688b1e8, 0xc002587040, 0xc00143c1b0, 0xc0020b0368, 0x6, 0x8, 0xc001fd4848, 0x2, 0x2, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:277 +0x4a9
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed.func1.1(0x688b1e8, 0xc002587040, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:80 +0x145
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002ae41b0, 0xc0027f2b10)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 2056 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/idalloc.(*Allocator).start.func1(0x688b290, 0xc0008b0990)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/idalloc/id_alloc.go:151 +0x3cd
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b290, 0xc0008b0990, 0xc000c30b80, 0x0, 0xc002c98a80)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 2291 [IO wait]:
internal/poll.runtime_pollWait(0x7f32aa3ea958, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc002b0e998, 0x72, 0x700, 0x796, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc002b0e980, 0xc002960000, 0x796, 0x796, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc002b0e980, 0xc002960000, 0x796, 0x796, 0xc002960005, 0xc00252f418, 0xc002353828)
	/usr/local/go/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc002c4fe48, 0xc002960000, 0x796, 0x796, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:183 +0x91
github.com/cockroachdb/cmux.(*MuxConn).Read(0xc002b4f020, 0xc002960000, 0x796, 0x796, 0x43995b, 0xc00205aa38, 0x18)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux/cmux.go:218 +0x11c
crypto/tls.(*atLeastReader).Read(0xc00205aa38, 0xc002960000, 0x796, 0x796, 0xc002353938, 0xc000300000, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:776 +0x63
bytes.(*Buffer).ReadFrom(0xc002af1078, 0x67eb580, 0xc00205aa38, 0x4376e5, 0x49b90e0, 0x4d913a0)
	/usr/local/go/src/bytes/buffer.go:204 +0xbe
crypto/tls.(*Conn).readFromUntil(0xc002af0e00, 0x7f32ab87f1a8, 0xc002b4f020, 0x5, 0xc002b4f020, 0x7f32adbeefff)
	/usr/local/go/src/crypto/tls/conn.go:798 +0xf3
crypto/tls.(*Conn).readRecordOrCCS(0xc002af0e00, 0x0, 0x0, 0xc0034e3260)
	/usr/local/go/src/crypto/tls/conn.go:605 +0x115
crypto/tls.(*Conn).readRecord(...)
	/usr/local/go/src/crypto/tls/conn.go:573
crypto/tls.(*Conn).Read(0xc002af0e00, 0xc00296e000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1276 +0x165
bufio.(*Reader).Read(0xc002755e60, 0xc00266dc38, 0x9, 0x9, 0xc00205aa20, 0xc002353ce0, 0x1)
	/usr/local/go/src/bufio/bufio.go:227 +0x222
io.ReadAtLeast(0x67eb320, 0xc002755e60, 0xc00266dc38, 0x9, 0x9, 0x9, 0x1, 0xc00205aa20, 0x0)
	/usr/local/go/src/io/io.go:328 +0x87
io.ReadFull(...)
	/usr/local/go/src/io/io.go:347
golang.org/x/net/http2.readFrameHeader(0xc00266dc38, 0x9, 0x9, 0x67eb320, 0xc002755e60, 0x0, 0xc000000000, 0x58, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2/frame.go:237 +0x89
golang.org/x/net/http2.(*Framer).ReadFrame(0xc00266dc00, 0xc002767590, 0x91a1b40, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2/frame.go:492 +0xa5
google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc002976000, 0xc002767710, 0x5779370)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_server.go:541 +0x9b
google.golang.org/grpc.(*Server).serveStreams(0xc000adcfc0, 0x68f45b8, 0xc002976000)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/server.go:925 +0x103
google.golang.org/grpc.(*Server).handleRawConn.func1(0xc000adcfc0, 0x68f45b8, 0xc002976000, 0xc0021b9870, 0xf)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/server.go:875 +0x3f
created by google.golang.org/grpc.(*Server).handleRawConn
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/server.go:874 +0x5f4

goroutine 2280 [select, 4 minutes]:
google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc00245d130)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:71 +0xb2
created by google.golang.org/grpc.newCCBalancerWrapper
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:62 +0x1de

goroutine 3227 [semacquire, 4 minutes]:
sync.runtime_Semacquire(0xc002ae41c0)
	/usr/local/go/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc002ae41b8)
	/usr/local/go/src/sync/waitgroup.go:130 +0x65
golang.org/x/sync/errgroup.(*Group).Wait(0xc002ae41b0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:40 +0x31
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait(0xc002ae41b0, 0x688b1e8, 0xc002587040, 0xc002b57260, 0xc002587040)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:142 +0x57
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed(0xc00080c000, 0x688b1e8, 0xc002587000, 0xc001fd4840, 0x2, 0x8, 0xc001fd4848, 0x2, 0x2, 0x16b6985d65f3cd0b, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:93 +0x4f8
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed.rangefeedFactory.Run.func1(0x688b1e8, 0xc002587000, 0xc000a5c5a0, 0x5776f70)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed/physical_kv_feed.go:77 +0xca
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002ae40f0, 0xc002ae4150)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 2257 [select]:
google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc00274eff0, 0x1, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:406 +0xff
google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc002755ec0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:533 +0x1dd
google.golang.org/grpc/internal/transport.NewServerTransport.func2(0xc002976000)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_server.go:304 +0xd7
created by google.golang.org/grpc/internal/transport.NewServerTransport
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_server.go:301 +0x11b0

goroutine 3226 [select]:
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed.(*rangefeed).addEventsToBuffer(0xc002f0c000, 0x688b1e8, 0xc002587000, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/kvfeed/physical_kv_feed.go:86 +0xdd
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002ae40f0, 0xc002ae4120)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 2270 [IO wait]:
internal/poll.runtime_pollWait(0x7f32aa3eaa40, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc002b0e918, 0x72, 0x800, 0x8d8, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc002b0e900, 0xc0011fed00, 0x8d8, 0x8d8, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc002b0e900, 0xc0011fed00, 0x8d8, 0x8d8, 0x8d3, 0xc0011fed00, 0x5)
	/usr/local/go/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc002c4fe38, 0xc0011fed00, 0x8d8, 0x8d8, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:183 +0x91
crypto/tls.(*atLeastReader).Read(0xc00000f2f0, 0xc0011fed00, 0x8d8, 0x8d8, 0x8d3, 0xc000300400, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:776 +0x63
bytes.(*Buffer).ReadFrom(0xc002af0cf8, 0x67eb580, 0xc00000f2f0, 0x4376e5, 0x49b90e0, 0x4fc4ec0)
	/usr/local/go/src/bytes/buffer.go:204 +0xbe
crypto/tls.(*Conn).readFromUntil(0xc002af0a80, 0x67f5d60, 0xc002c4fe38, 0x5, 0xc002c4fe38, 0xc002102958)
	/usr/local/go/src/crypto/tls/conn.go:798 +0xf3
crypto/tls.(*Conn).readRecordOrCCS(0xc002af0a80, 0x0, 0x0, 0xc002d8b860)
	/usr/local/go/src/crypto/tls/conn.go:605 +0x115
crypto/tls.(*Conn).readRecord(...)
	/usr/local/go/src/crypto/tls/conn.go:573
crypto/tls.(*Conn).Read(0xc002af0a80, 0xc00301a000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1276 +0x165
bufio.(*Reader).Read(0xc0030182a0, 0xc002ffe818, 0x9, 0x9, 0x0, 0xc002102c90, 0x6842cd8)
	/usr/local/go/src/bufio/bufio.go:227 +0x222
io.ReadAtLeast(0x67eb320, 0xc0030182a0, 0xc002ffe818, 0x9, 0x9, 0x9, 0xc002114510, 0x0, 0x0)
	/usr/local/go/src/io/io.go:328 +0x87
io.ReadFull(...)
	/usr/local/go/src/io/io.go:347
golang.org/x/net/http2.readFrameHeader(0xc002ffe818, 0x9, 0x9, 0x67eb320, 0xc0030182a0, 0x0, 0xc000000000, 0xc002114480, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2/frame.go:237 +0x89
golang.org/x/net/http2.(*Framer).ReadFrame(0xc002ffe7e0, 0xc0021144b0, 0x91a1b40, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2/frame.go:492 +0xa5
google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc002d8b860)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1495 +0x1a5
created by google.golang.org/grpc/internal/transport.newHTTP2Client
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:355 +0xdd1

goroutine 3238 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).outputLoop(0xc00240afc0, 0x688b1e8, 0xc002531180, 0xc002531180, 0xc002c2ec40)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:267 +0x1b8
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).runOutputLoop(0xc00240afc0, 0x688b1e8, 0xc002531180, 0x2e)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:289 +0xd4
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run.func1(0x688b1e8, 0xc002531040)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:275 +0x6d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2(0xc000a5c5a0, 0x688b1e8, 0xc002531040, 0xc000c30b80, 0x0, 0xc002a68000)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:491 +0x103
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:481 +0x2ad

goroutine 3239 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).rangeFeedWithRangeID(0xc001514a80, 0x2f, 0xc001e71100, 0x68e13e8, 0xc002553e80, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_rangefeed.go:244 +0x670
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).RangeFeed(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_rangefeed.go:142
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).RangeFeed(0xc00026c800, 0xc001e71100, 0x68e13e8, 0xc002553e80, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:2661 +0x136
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).RangeFeed(0xc0002d02a0, 0xc001e71100, 0x68e13e8, 0xc002553e80, 0xc001690f68)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/stores.go:216 +0xce
github.com/cockroachdb/cockroach/pkg/server.(*Node).RangeFeed(0xc000238a00, 0xc001e71100, 0x68e13e8, 0xc002553e80, 0xc001690f68, 0x1ab3fba)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:1090 +0x54
github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.RangeFeed.func1(0xc002c2ec70, 0x68f1c78, 0xc000238a00, 0xc001e71100, 0x688b1e8, 0xc002531200, 0xc002897800, 0xc002897860)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:617 +0xc5
created by github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.RangeFeed
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:615 +0x12e

goroutine 2186 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*teeGroup).tee.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/testfeed_test.go:982 +0x1b8
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002a36fc0, 0xc00266bb00)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 3037 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*teeGroup).tee.func1(0xc002754790, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/testfeed_test.go:982 +0x1b8
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002a36fc0, 0xc002726690)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 2189 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*teeGroup).tee.func1(0xc000c30b80, 0x5777018)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/testfeed_test.go:982 +0x1b8
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc002a36fc0, 0xc00266bcb0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66

goroutine 3038 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*kafkaSink).workerLoop(0xc001f09e00)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/sink_kafka.go:338 +0x158
created by github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*kafkaSink).start
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/sink_kafka.go:164 +0x8b

goroutine 3235 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).rangeFeedWithRangeID(0xc001029500, 0x2e, 0xc001e71000, 0x68e13e8, 0xc002553d40, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_rangefeed.go:244 +0x670
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).RangeFeed(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_rangefeed.go:142
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).RangeFeed(0xc00026c800, 0xc001e71000, 0x68e13e8, 0xc002553d40, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:2661 +0x136
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).RangeFeed(0xc0002d02a0, 0xc001e71000, 0x68e13e8, 0xc002553d40, 0xc0014d3768)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/stores.go:216 +0xce
github.com/cockroachdb/cockroach/pkg/server.(*Node).RangeFeed(0xc000238a00, 0xc001e71000, 0x68e13e8, 0xc002553d40, 0xc0014d3768, 0x1ab3fba)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:1090 +0x54
github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.RangeFeed.func1(0xc002c2ebc0, 0x68f1c78, 0xc000238a00, 0xc001e71000, 0x688b1e8, 0xc002531000, 0xc0028972c0, 0xc0028973e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:617 +0xc5
created by github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.RangeFeed
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:615 +0x12e


ERROR: exit status 2
```
</details>



Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cdc: changefeed TeamCity flakiness #72706

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

cdc: changefeed TeamCity flakiness #72706

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions