Skip to content

upgrade/upgrades: TestJSONForwardingIndexes failed #107169

@cockroach-teamcity

Description

@cockroach-teamcity

upgrade/upgrades.TestJSONForwardingIndexes failed with artifacts on master @ 208ba72d61e334380be3f0bcf7ab1ecb3fae6ebd:

I230719 14:18:56.610677 14872 upgrade/upgrademanager/manager.go:657  [T1,n1,client=127.0.0.1:51566,hostssl,user=root,migration-mgr] 2938  executing operation bump-cluster-version=1000023.1--1
I230719 14:18:56.610924 14872 upgrade/upgradecluster/cluster.go:114  [T1,n1,client=127.0.0.1:51566,hostssl,user=root,migration-mgr] 2939  executing bump-cluster-version=1000023.1--1 on nodes n{1}
I230719 14:18:56.611166 36684 server/migration.go:150  [T1,n1,bump-cluster-version] 2940  active cluster version setting is now 1000023.1--1 (up from 1000022.2-102)
I230719 14:18:56.611425 14872 upgrade/upgrademanager/manager.go:657  [T1,n1,client=127.0.0.1:51566,hostssl,user=root,migration-mgr] 2941  executing operation validate-cluster-version=1000023.1
I230719 14:18:56.611656 14872 upgrade/upgradecluster/cluster.go:114  [T1,n1,client=127.0.0.1:51566,hostssl,user=root,migration-mgr] 2942  executing validate-cluster-version=1000023.1 on nodes n{1}
I230719 14:18:56.612041 14872 upgrade/upgrademanager/manager.go:657  [T1,n1,client=127.0.0.1:51566,hostssl,user=root,migration-mgr] 2943  executing operation bump-cluster-version=1000023.1
I230719 14:18:56.612271 14872 upgrade/upgradecluster/cluster.go:114  [T1,n1,client=127.0.0.1:51566,hostssl,user=root,migration-mgr] 2944  executing bump-cluster-version=1000023.1 on nodes n{1}
I230719 14:18:56.612479 36702 server/migration.go:150  [T1,n1,bump-cluster-version] 2945  active cluster version setting is now 1000023.1 (up from 1000023.1--1)
I230719 14:18:56.614762 14872 util/log/event_log.go:32  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2946 ={"Timestamp":1689776325275326498,"EventType":"set_cluster_setting","Statement":"SET CLUSTER SETTING version = $1","Tag":"SET CLUSTER SETTING","User":"root","PlaceholderValues":["'1000023.1'"],"SettingName":"version","Value":"1000023.1"}
I230719 14:18:56.615162 14872 sql/schemachanger/scbuild/build.go:55  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2947  building declarative schema change targets for CREATE INDEX
W230719 14:18:56.618501 14872 sql/schemachanger/scbuild/internal/scbuildstmt/create_index.go:393  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2948  failed building declarative schema change targets for CREATE INDEX with error: unimplemented: column j is of type jsonb and thus is not indexable
W230719 14:18:56.618501 14872 sql/schemachanger/scbuild/internal/scbuildstmt/create_index.go:393  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2948 +HINT: You have attempted to use a feature that is not yet implemented.
W230719 14:18:56.618501 14872 sql/schemachanger/scbuild/internal/scbuildstmt/create_index.go:393  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2948 +See: https://go.crdb.dev/issue-v/35730/v23.2
I230719 14:18:56.618947 14872 sql/schemachanger/scbuild/build.go:55  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2949  building declarative schema change targets for CREATE INDEX
W230719 14:18:56.623067 14872 sql/schemachanger/scbuild/internal/scbuildstmt/create_index.go:393  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2950  failed building declarative schema change targets for CREATE INDEX with error: unimplemented: column crdb_internal_idx_expr is of type jsonb and thus is not indexable
W230719 14:18:56.623067 14872 sql/schemachanger/scbuild/internal/scbuildstmt/create_index.go:393  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2950 +HINT: You have attempted to use a feature that is not yet implemented.
W230719 14:18:56.623067 14872 sql/schemachanger/scbuild/internal/scbuildstmt/create_index.go:393  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2950 +See: https://go.crdb.dev/issue-v/35730/v23.2
I230719 14:18:56.627308 14923 9@sql/pgwire/conn.go:367  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2951  could not parse simple query: CREATE TABLE test.tbl_json_secondary_unique(
I230719 14:18:56.627308 14923 9@sql/pgwire/conn.go:367  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2951 +    j JSONB,
I230719 14:18:56.627308 14923 9@sql/pgwire/conn.go:367  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2951 +    UNIQUE tbl_json_secondary_uidx (j)
I230719 14:18:56.627308 14923 9@sql/pgwire/conn.go:367  [T1,n1,client=127.0.0.1:51566,hostssl,user=root] 2951 +)
I230719 14:18:56.639007 14872 upgrade/upgrademanager/manager.go:397  [T1,n1,client=127.0.0.1:51566,hostssl,user=root,migration-mgr] 2952  migrating cluster from 1000023.1 to 1000023.1-14 (stepping through [1000023.1-2 1000023.1-4 1000023.1-6 1000023.1-8 1000023.1-10 1000023.1-12 1000023.1-14])
I230719 14:18:56.639094 14872 upgrade/upgrademanager/manager.go:657  [T1,n1,client=127.0.0.1:51566,hostssl,user=root,migration-mgr] 2953  executing operation validate-cluster-version=1000023.1-14
I230719 14:18:56.639373 14872 upgrade/upgradecluster/cluster.go:114  [T1,n1,client=127.0.0.1:51566,hostssl,user=root,migration-mgr] 2954  executing validate-cluster-version=1000023.1-14 on nodes n{1}
W230719 14:18:56.663013 14872 upgrade/upgrademanager/manager.go:370  [T1,n1,client=127.0.0.1:51566,hostssl,user=root,migration-mgr] 2955  error encountered during version upgrade: verifying precondition for version 1000023.1-2: query-invalid_objects: crdb-internal-jobs-table: system-jobs-scan: column "job_type" does not exist
    json_forward_indexes_test.go:147: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/upgrade/upgrades_test/pkg/upgrade/upgrades/json_forward_indexes_test.go:147
        	Error:      	Received unexpected error:
        	            	pq: verifying precondition for version 1000023.1-2: query-invalid_objects: crdb-internal-jobs-table: system-jobs-scan: column "job_type" does not exist
        	Test:       	TestJSONForwardingIndexes
I230719 14:18:56.663988 13364 testutils/testcluster/testcluster.go:148  [-] 2956  TestCluster quiescing nodes
I230719 14:18:56.664127 14098 sql/stats/automatic_stats.go:572  [T1,n1] 2957  quiescing auto stats refresher
I230719 14:18:56.664154 14099 sql/stats/automatic_stats.go:624  [T1,n1] 2958  quiescing stats garbage collector
W230719 14:18:56.664163 36417 kv/kvserver/intentresolver/intent_resolver.go:826  [-] 2959  failed to gc transaction record: could not GC completed transaction anchored at /Table/15/1/883772412356329473/0: node unavailable; try another peer
I230719 14:18:56.664272 13610 server/start_listen.go:103  [T1,n1] 2960  server shutting down: instructing cmux to stop accepting
W230719 14:18:56.664716 13867 sql/sqlliveness/slinstance/slinstance.go:334  [T1,n1] 2961  exiting heartbeat loop
W230719 14:18:56.664741 13867 sql/sqlliveness/slinstance/slinstance.go:321  [T1,n1] 2962  exiting heartbeat loop with error: node unavailable; try another peer
E230719 14:18:56.664759 13867 server/server_sql.go:515  [T1,n1] 2963  failed to run update of instance with new session ID: node unavailable; try another peer
I230719 14:18:56.664805 14319 jobs/registry.go:1606  [T1,n1] 2964  AUTO SPAN CONFIG RECONCILIATION job 883772374920265729: stepping through state succeeded
E230719 14:18:56.664820 27904 jobs/registry.go:1004  [T1,n1] 2965  error getting live session: node unavailable; try another peer
W230719 14:18:56.682596 31847 jobs/adopt.go:534  [T1,n1] 2966  could not clear job claim: clear-job-claim: node unavailable; try another peer
W230719 14:18:56.682721 26450 jobs/adopt.go:534  [T1,n1] 2967  could not clear job claim: clear-job-claim: failed to send RPC: sending to all replicas failed; last error: ba: Scan [/Table/15/1/102,/Table/15/1/103), [txn: 94b87cd5], [can-forward-ts] RPC error: node unavailable; try another peer
W230719 14:18:56.682830 26457 jobs/adopt.go:534  [T1,n1] 2968  could not clear job claim: clear-job-claim: node unavailable; try another peer
W230719 14:18:56.685108 35783 jobs/adopt.go:534  [T1,n1] 2969  could not clear job claim: clear-job-claim: node unavailable; try another peer
W230719 14:18:56.688088 30556 jobs/adopt.go:534  [T1,n1] 2970  could not clear job claim: clear-job-claim: node unavailable; try another peer
W230719 14:18:56.697942 29501 jobs/adopt.go:534  [T1,n1] 2971  could not clear job claim: clear-job-claim: node unavailable; try another peer
I230719 14:18:56.701753 13364 server/server_controller_orchestration.go:263  [T1,n1] 2972  server controller shutting down ungracefully
I230719 14:18:56.701799 13364 server/server_controller_orchestration.go:274  [T1,n1] 2973  waiting for tenant servers to report stopped
W230719 14:18:56.701894 13364 server/server_sql.go:1675  [T1,n1] 2974  server shutdown without a prior graceful drain
--- FAIL: TestJSONForwardingIndexes (11.85s)
Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/release-eng

This test on roachdash | Improve this report!

Jira issue: CRDB-29932

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions