sql/catalog/lease: TestTxnObeysTableModificationTime failed [liveness failure caused jobs failure]

sql/catalog/lease.TestTxnObeysTableModificationTime [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_StressBazel/10969631?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_StressBazel/10969631?buildTab=artifacts#/) on release-23.1 @ [207e4959d30012b2887f31f32d3fdaea630496b9](https://github.com/cockroachdb/cockroach/commits/207e4959d30012b2887f31f32d3fdaea630496b9):


```
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	github.com/cockroachdb/cockroach/pkg/kv/db.go:927
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | github.com/cockroachdb/cockroach/pkg/kv.(*DB).Txn
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	github.com/cockroachdb/cockroach/pkg/kv/db.go:902
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | github.com/cockroachdb/cockroach/pkg/sql.(*InternalDB).txn
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	github.com/cockroachdb/cockroach/pkg/sql/internal.go:1686
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | github.com/cockroachdb/cockroach/pkg/sql.(*InternalDB).Txn
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	github.com/cockroachdb/cockroach/pkg/sql/internal.go:1621
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | github.com/cockroachdb/cockroach/pkg/sql/gcjob.persistProgress
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	github.com/cockroachdb/cockroach/pkg/sql/gcjob/gc_job_utils.go:281
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | github.com/cockroachdb/cockroach/pkg/sql/gcjob.schemaChangeGCResumer.deleteDataAndWaitForGC
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	github.com/cockroachdb/cockroach/pkg/sql/gcjob/gc_job.go:333
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | github.com/cockroachdb/cockroach/pkg/sql/gcjob.schemaChangeGCResumer.Resume
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	github.com/cockroachdb/cockroach/pkg/sql/gcjob/gc_job.go:314
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine.func2
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	github.com/cockroachdb/cockroach/pkg/jobs/registry.go:1628
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	github.com/cockroachdb/cockroach/pkg/jobs/registry.go:1629
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).runJob
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	github.com/cockroachdb/cockroach/pkg/jobs/adopt.go:474
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).resumeJob.func2
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	github.com/cockroachdb/cockroach/pkg/jobs/adopt.go:388
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | [...repeated from below...]
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 +Wraps: (6) failed to read query result
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 +Wraps: (7) candidate pg code: 57014
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 +Wraps: (8) attached stack trace
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + -- stack trace:
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | github.com/cockroachdb/cockroach/pkg/util/cancelchecker.init
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	github.com/cockroachdb/cockroach/pkg/util/cancelchecker/cancel_checker.go:80
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | runtime.doInit
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	GOROOT/src/runtime/proc.go:6349
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | runtime.doInit
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	GOROOT/src/runtime/proc.go:6326
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | runtime.doInit
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	GOROOT/src/runtime/proc.go:6326
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | runtime.doInit
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	GOROOT/src/runtime/proc.go:6326
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | runtime.doInit
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	GOROOT/src/runtime/proc.go:6326
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | runtime.doInit
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	GOROOT/src/runtime/proc.go:6326
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | runtime.main
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	GOROOT/src/runtime/proc.go:233
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | runtime.goexit
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 + | 	GOROOT/src/runtime/asm_amd64.s:1594
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 +Wraps: (9) query execution canceled
W230719 09:54:18.444261 93470 sql/gcjob/gc_job_utils.go:299 [T1,n1,job=SCHEMA CHANGE GC id=883720383116476417] 5433 +Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.withPrefix (7) *pgerror.withCandidateCode (8) *withstack.withStack (9) *errutil.leafError
I230719 09:54:18.446704 81233 server/server_controller_orchestration.go:263 [T1,n1] 5434 server controller shutting down ungracefully
I230719 09:54:18.446745 81233 server/server_controller_orchestration.go:274 [T1,n1] 5435 waiting for tenant servers to report stopped
W230719 09:54:18.446814 81233 server/server_sql.go:1704 [T1,n1] 5436 server shutdown without a prior graceful drain
--- FAIL: TestTxnObeysTableModificationTime (1.79s)
```
Parameters: <code>TAGS=bazel,gss</code>

<details><summary>Help</summary>


See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

</details>
/cc @cockroachdb/sql-foundations


[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestTxnObeysTableModificationTime.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)



Jira issue: CRDB-29922

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sql/catalog/lease: TestTxnObeysTableModificationTime failed [liveness failure caused jobs failure] #107159

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

sql/catalog/lease: TestTxnObeysTableModificationTime failed [liveness failure caused jobs failure] #107159

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions