Skip to content

ccl/testccl/workload/schemachange: TestWorkload failed #62503

@cockroach-teamcity

Description

@cockroach-teamcity

(ccl/testccl/workload/schemachange).TestWorkload failed on master@ab7362161d5e3b88f4730db94b95c5926b5eb3af:

1, 29, rangelog, 13
1, 29, replication_constraint_stats, 25
1, 29, replication_critical_localities, 26
1, 29, replication_stats, 27
1, 29, reports_meta, 28
1, 29, role_members, 23
1, 29, role_options, 33
1, 29, scheduled_jobs, 37
1, 29, settings, 6
1, 29, sqlliveness, 39
1, 29, statement_bundle_chunks, 34
1, 29, statement_diagnostics, 36
1, 29, statement_diagnostics_requests, 35
1, 29, table_statistics, 20
1, 29, tenants, 8
1, 29, ui, 14
1, 29, users, 4
1, 29, web_sessions, 19
1, 29, zones, 5
50, 0, public, 29
51, 0, public, 29
52, 0, public, 29
52, 0, schema21, 62
52, 29, _crdb_internal_region, 64
52, 29, crdb_internal_region, 63
I210323 22:26:12.630571 10325 9@util/log/event_log.go:32  [n1,client=127.0.0.1:40976,hostssl,user=root,intExec=backup-lookup-tenant] 3159 ={"Timestamp":1616538372600117161,"EventType":"query_execute","Statement":"SELECT id, active, info FROM \"\".system.tenants","User":"root","ApplicationName":"$ internal-backup-lookup-tenant","ExecMode":"exec-internal","Age":8.822355,"FullTableScan":true}
I210323 22:26:12.642492 10326 9@util/log/event_log.go:32  [n1,client=127.0.0.1:40976,hostssl,user=root,intExec=job-row-insert] 3160 ={"Timestamp":1616538372632047538,"EventType":"query_execute","Statement":"INSERT INTO \"\".system.jobs(id, status, payload, progress, claim_session_id, claim_instance_id) VALUES ($1, $2, $3, $4, $5, $6)","User":"root","ApplicationName":"$ internal-job-row-insert","PlaceholderValues":["643786252720635905","'running'","'\\x0a364241434b555020444154414241534520736368656d616368616e676520544f20276e6f64656c6f63616c3a2f2f302f6261636b7570271204726f6f743204343e3f4052240a00120a08dcb2cdc5dca2c6b7161a146e6f64656c6f63616c3a2f2f302f6261636b7570'","'\\x10f0dbdddbbac7ef025200'","'\\x16ba01fba0324fdeb8c6f38ffbc5a1ae'","1"],"ExecMode":"exec-internal","NumRows":1,"Age":10.106648}
I210323 22:26:12.684922 10349 9@util/log/event_log.go:32  [n1,client=127.0.0.1:40976,hostssl,user=root,intExec=log-job] 3161 ={"Timestamp":1616538372671621823,"EventType":"query_execute","Statement":"SELECT status, payload, progress, claim_session_id FROM \"\".system.jobs WHERE id = $1 FOR UPDATE","User":"root","ApplicationName":"$ internal-log-job","PlaceholderValues":["643786252720635905"],"ExecMode":"exec-internal","NumRows":1,"Age":13.052351}
I210323 22:26:12.700214 10351 9@util/log/event_log.go:32  [n1,client=127.0.0.1:40976,hostssl,user=root,intExec=job-update] 3162 ={"Timestamp":1616538372671621823,"EventType":"query_execute","Statement":"UPDATE \"\".system.jobs SET status = $2, payload = $3 WHERE id = $1","User":"root","ApplicationName":"$ internal-job-update","PlaceholderValues":["643786252720635905","'running'","'\\x0a364241434b555020444154414241534520736368656d616368616e676520544f20276e6f64656c6f63616c3a2f2f302f6261636b7570271204726f6f7418b983e1dbbac7ef023204343e3f4052240a00120a08dcb2cdc5dca2c6b7161a146e6f64656c6f63616c3a2f2f302f6261636b7570'"],"ExecMode":"exec-internal","NumRows":1,"Age":13.632049}
I210323 22:26:12.709359 10251 jobs/registry.go:1175  [n1] 3163  BACKUP job 643786252720635905: stepping through state running with error: <nil>
I210323 22:26:12.758856 10356 9@util/log/event_log.go:32  [n1,job=643786252720635905,intExec=lookup-schedule-info] 3164 ={"Timestamp":1616538372745548302,"EventType":"query_execute","Statement":"SELECT created_by_id FROM \"\".system.jobs WHERE (id = $1) AND (created_by_type = $2)","User":"node","ApplicationName":"$ internal-lookup-schedule-info","PlaceholderValues":["643786252720635905","'crdb_schedule'"],"ExecMode":"exec-internal","Age":13.064578}
I210323 22:26:12.760043 10251 jobs/registry.go:1175  [n1] 3165  BACKUP job 643786252720635905: stepping through state succeeded with error: <nil>
I210323 22:26:12.773568 10357 9@util/log/event_log.go:32  [n1,intExec=log-job] 3166 ={"Timestamp":1616538372760387669,"EventType":"query_execute","Statement":"SELECT status, payload, progress, claim_session_id FROM \"\".system.jobs WHERE id = $1 FOR UPDATE","User":"root","ApplicationName":"$ internal-log-job","PlaceholderValues":["643786252720635905"],"ExecMode":"exec-internal","NumRows":1,"Age":12.981474}
I210323 22:26:12.790549 10359 9@util/log/event_log.go:32  [n1,intExec=job-update] 3167 ={"Timestamp":1616538372760387669,"EventType":"query_execute","Statement":"UPDATE \"\".system.jobs SET status = $2, payload = $3, progress = $4 WHERE id = $1","User":"root","ApplicationName":"$ internal-job-update","PlaceholderValues":["643786252720635905","'succeeded'","'\\x0a364241434b555020444154414241534520736368656d616368616e676520544f20276e6f64656c6f63616c3a2f2f302f6261636b7570271204726f6f7418b983e1dbbac7ef022080b8e6dbbac7ef023204343e3f4052240a00120a08dcb2cdc5dca2c6b7161a146e6f64656c6f63616c3a2f2f302f6261636b7570'","'\\x0d0000803f10c4c6e5dbbac7ef025200'"],"ExecMode":"exec-internal","NumRows":1,"Age":15.298867}
I210323 22:26:12.801368 6349 9@util/log/event_log.go:32  [n1,client=127.0.0.1:40976,hostssl,user=root] 3168 ={"Timestamp":1616538372600117161,"EventType":"query_execute","Statement":"BACKUP DATABASE schemachange TO 'nodelocal://0/backup'","User":"root","ExecMode":"exec","NumRows":1,"Age":201.52194,"TxnCounter":7}
    schema_change_external_test.go:81: backup in /go/src/github.com/cockroachdb/cockroach/artifacts/TestWorkload177599007
I210323 22:26:12.802946 13 testutils/testcluster/testcluster.go:101  [-] 3169  TestCluster quiescing nodes
E210323 22:26:12.804834 10286 kv/kvclient/kvcoord/txn_interceptor_committer.go:452  [-] 3170  making txn commit explicit failed for "unnamed" meta={id=6e1198b5 pri=0.00438211 epo=0 ts=1616538372.760387669,0 min=1616538372.760387669,0 seq=6} lock=true stat=STAGING rts=1616538372.760387669,0 wto=false gul=1616538373.260387669,0 int=1 ifw=5: result is ambiguous (server shutdown)
W210323 22:26:12.807100 10354 kv/kvserver/intentresolver/intent_resolver.go:758  [-] 3171  failed to gc transaction record: could not GC completed transaction anchored at /Table/15/1/643786252720635905: node unavailable; try another peer
W210323 22:26:12.808725 1462 sql/sqlliveness/slinstance/slinstance.go:183  [n2] 3172  exiting heartbeat loop
W210323 22:26:12.811784 1463 jobs/registry.go:729  [-] 3173  canceling all adopted jobs due to stopper quiescing
W210323 22:26:12.814841 1801 jobs/registry.go:729  [-] 3174  canceling all adopted jobs due to stopper quiescing
W210323 22:26:12.816624 1970 kv/kvserver/closedts/sidetransport/receiver.go:125  [n2] 3175  closed timestamps side-transport connection dropped from node: 1
W210323 22:26:12.818102 365 sql/sqlliveness/slinstance/slinstance.go:183  [n1] 3176  exiting heartbeat loop
W210323 22:26:12.818882 1800 sql/sqlliveness/slinstance/slinstance.go:183  [n3] 3177  exiting heartbeat loop
W210323 22:26:12.823340 366 jobs/registry.go:729  [-] 3178  canceling all adopted jobs due to stopper quiescing
W210323 22:26:12.823975 1195 2@gossip/gossip.go:1491  [n2] 3179  no incoming or outgoing connections
W210323 22:26:12.831381 2137 kv/kvserver/closedts/sidetransport/receiver.go:125  [n3] 3180  closed timestamps side-transport connection dropped from node: 1
I210323 22:26:12.881459 314 kv/kvclient/kvcoord/transport_race.go:106  [-] 3181  transport race promotion: ran 65 iterations on up to 3625 requests
--- FAIL: TestWorkload (19.66s)

More

Parameters:

  • GOFLAGS=-json
make stressrace TESTS=TestWorkload PKG=./pkg/ccl/testccl/workload/schemachange TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

Metadata

Metadata

Assignees

Labels

C-test-failureBroken test (automatically or manually discovered).O-robotOriginated from a bot.branch-masterFailures and bugs on the master branch.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions