Skip to content

sql: add ColumnGeneratedAsIdentity as a new schema change element#156547

Merged
craig[bot] merged 2 commits intocockroachdb:masterfrom
shghasemi:genid-elem
Nov 6, 2025
Merged

sql: add ColumnGeneratedAsIdentity as a new schema change element#156547
craig[bot] merged 2 commits intocockroachdb:masterfrom
shghasemi:genid-elem

Conversation

@shghasemi
Copy link
Copy Markdown
Contributor

Previously, GeneratedAsIdentity was part of the Column element.
Creating a new schema change element will allow the declarative schema changer
to add/remove/update such columns without dropping the Column element.

Epic: CRDB-31283

Part of: #142918

Release note: None

@shghasemi shghasemi requested a review from a team as a code owner October 30, 2025 15:03
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@shghasemi shghasemi self-assigned this Oct 30, 2025
@shghasemi shghasemi marked this pull request as draft October 30, 2025 17:15
@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented Oct 31, 2025

Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@shghasemi shghasemi force-pushed the genid-elem branch 4 times, most recently from 4d93d2c to 880e3cb Compare November 4, 2025 15:15
@shghasemi shghasemi changed the title sql: add ColumnGeneratedAsIdentity update test output sql: add ColumnGeneratedAsIdentity as a new schema change element Nov 4, 2025
@shghasemi shghasemi force-pushed the genid-elem branch 2 times, most recently from f5ff419 to 9fc3c0c Compare November 4, 2025 19:25
@shghasemi shghasemi marked this pull request as ready for review November 4, 2025 20:14
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Nov 4, 2025

Potential Bug(s) Detected

The three-stage Claude Code analysis has identified potential bug(s) in this PR that may warrant investigation.

Next Steps:
Please review the detailed findings in the workflow run.

Note: When viewing the workflow output, scroll to the bottom to find the Final Analysis Summary.

After you review the findings, please tag the issue as follows:

  • If the detected issue is real or was helpful in any way, please tag the issue with O-AI-Review-Real-Issue-Found
  • If the detected issue was not helpful in any way, please tag the issue with O-AI-Review-Not-Helpful

@github-actions github-actions bot added the o-AI-Review-Potential-Issue-Detected AI reviewer found potential issue. Never assign manually—auto-applied by GH action only. label Nov 4, 2025
Copy link
Copy Markdown
Collaborator

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @shghasemi! A couple of minor issues

@fqazi reviewed 25 of 47 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @shghasemi)


pkg/sql/schemachanger/scplan/internal/opgen/opgen_column_generated_as_identity.go line 25 at r1 (raw file):

				}),
			),
			equiv(scpb.Status_WRITE_ONLY),

Let's have a comment explaining why we need this, which is only for the migration case.


pkg/sql/schemachanger/screl/attr.go line 120 at r1 (raw file):

	// GeneratedAsIdentityType is the type for a generated as identity column.
	// It's value must be in catpb.GeneratedAsIdentityType.
	GeneratedAsIdentityType

The rules are not using these fields. We don't need any new attrs here.


pkg/sql/schemachanger/screl/attr.go line 335 at r1 (raw file):

		rel.EntityAttr(DescID, "TableID"),
		rel.EntityAttr(ColumnID, "ColumnID"),
		rel.EntityAttr(GeneratedAsIdentityType, "Type"),

Remove the type and sequence opts unless the rules need them.


pkg/sql/schemachanger/scpb/migration_test.go line 143 at r1 (raw file):

// TestDeprecatedColumnGeneratedAsIdentity will ensure that a ColumnGeneratedAsIdentity
// element is added.
func TestDeprecatedColumnGeneratedAsIdentity(t *testing.T) {

Nice test!


pkg/sql/schemachanger/scplan/internal/rules/release_25_4/helpers.go line 257 at r1 (raw file):

		return true
	case *scpb.ColumnGeneratedAsIdentity:
		return true

Leave the older rules the same.


pkg/sql/schemachanger/scplan/internal/rules/release_25_2/helpers.go line 215 at r1 (raw file):

		return true
	case *scpb.ColumnGeneratedAsIdentity:
		return true

Leave the older rules the same.


pkg/sql/schemachanger/rel/ordinal_set.go line 15 at r1 (raw file):

// ordinal is used to correlate attributes in a schema.
// It enables use of the ordinalSet.
type ordinal uint16

What is this change here?


pkg/sql/schemachanger/scplan/internal/rules/release_25_3/helpers.go line 221 at r1 (raw file):

		return true
	case *scpb.ColumnGeneratedAsIdentity:
		return true

Leave the older rules the same.


pkg/sql/schemachanger/scpb/migration.go line 82 at r1 (raw file):

	// Migrate GeneratedAsIdentity from column to a separate element
	if column := target.GetColumn(); column != nil {
		if version.IsActive(clusterversion.V26_1) &&

Nit: Lets just && this above to reduce nesting.

@shghasemi
Copy link
Copy Markdown
Contributor Author

shghasemi commented Nov 5, 2025

pkg/sql/schemachanger/screl/attr.go line 120 at r1 (raw file):

	// GeneratedAsIdentityType is the type for a generated as identity column.
	// It's value must be in catpb.GeneratedAsIdentityType.
	GeneratedAsIdentityType

The rules are not using these fields. We don't need any new attrs here.

Removed.

pkg/sql/schemachanger/rel/ordinal_set.go line 15 at r1 (raw file):

// ordinal is used to correlate attributes in a schema.
// It enables use of the ordinalSet.
type ordinal uint16

What is this change here?

This change is no longer needed. Adding two new attrs was pushing the number of attrs over 30, and this change increased the maximum number of attrs.

pkg/sql/schemachanger/scpb/migration.go line 82 at r1 (raw file):

	// Migrate GeneratedAsIdentity from column to a separate element
	if column := target.GetColumn(); column != nil {
		if version.IsActive(clusterversion.V26_1) &&

Nit: Lets just && this above to reduce nesting.

Done

pkg/sql/schemachanger/scplan/internal/rules/release_25_3/helpers.go line 221 at r1 (raw file):

		return true
	case *scpb.ColumnGeneratedAsIdentity:
		return true

Leave the older rules the same.

The following test fails without this change:
FAIL: //pkg/sql/schemachanger/scplan/internal/rules/release_25_3:release_25_3_test

Copy link
Copy Markdown
Collaborator

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fqazi reviewed 22 of 47 files at r1.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @shghasemi)


pkg/sql/schemachanger/scplan/internal/rules/release_25_3/helpers.go line 221 at r1 (raw file):

Previously, shghasemi wrote…

The following test fails without this change:
FAIL: //pkg/sql/schemachanger/scplan/internal/rules/release_25_3:release_25_3_test

The test is incorrectly using ForEachElementType instead of ForEachElementInActiveVersion


pkg/sql/schemachanger/rel/rel_test.go line 184 at r1 (raw file):

		F10, F11, F12, F13, F14, F15, F16, F17, F18, F19 *uint32
		F20, F21, F22, F23, F24, F25, F26, F27, F28, F29 *uint32
		F30, F31, F32, F33, F34, F35, F36, F37, F38, F39 *uint32

Lets revert this change too

Previously, TestRuleAssertions was using ForEachLementType instead of
ForEachElementInActiveVersion. This caused invalid test fails when
a new element was added in a newer version.

Epic: CRDB-31283

Release note: None
Copy link
Copy Markdown
Collaborator

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@fqazi reviewed 55 of 55 files at r2, 47 of 47 files at r3, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @shghasemi)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Nov 5, 2025

Potential Bug(s) Detected

The three-stage Claude Code analysis has identified potential bug(s) in this PR that may warrant investigation.

Next Steps:
Please review the detailed findings in the workflow run.

Note: When viewing the workflow output, scroll to the bottom to find the Final Analysis Summary.

After you review the findings, please tag the issue as follows:

  • If the detected issue is real or was helpful in any way, please tag the issue with O-AI-Review-Real-Issue-Found
  • If the detected issue was not helpful in any way, please tag the issue with O-AI-Review-Not-Helpful

Previously, GeneratedAsIdentity was part of the Column element.
Creating a new schema change element will allow the declarative schema changer
to add/remove/update such columns without dropping the Column element.

Epic: CRDB-31283

Part of cockroachdb#142918

Release note: None
@shghasemi
Copy link
Copy Markdown
Contributor Author

TFTR!

bors r+

@shghasemi shghasemi added the O-AI-Review-Real-Issue-Found AI reviewer found real issue label Nov 6, 2025
craig bot pushed a commit that referenced this pull request Nov 6, 2025
154467: roachprod: azure GetVMSpecs() implementation r=DarrylWong a=williamchoe3

Informs #146202 

Implementation for previously unimplemented interface method `GetVMSpecs()` for Azure. This will allow for cluster information to be fetched in roachtest via `FetchVMSpecs`

Also changed `azureIDPattern` to match the optional type name pairs that may exist in a fully qualified azure resource id. See comments for more details.  Previously, if these optional type name pairs existed, they were just a part of the `resourceName` field. Currently, I didn't see any of these child resource ids being parsed, but if we were to ever parse those in the future the previous implementation wouldn't make sense.
* Note: Although the regex selects child resource type and names in the last matching group, they are not saved to AzureId struct because they are not being used

For Azure, `client.Get(...)` allows you to pass in an optional `expander` parameter which seems to provide additional information for `"instanceView"` and runs an optionally passed in user script (passed in on vm creation) when `"userData"` is passed in.
*  `"userData"` doesn't seem relevant for us "retrieves the UserData property as part of the VM model view that was provided by the user during the VM Create/Update operation" https://learn.microsoft.com/en-us/rest/api/compute/virtual-machines/get?view=rest-compute-2025-04-01&tabs=HTTP
  *  "The `instanceView` of an Azure Virtual Machine (VM) provides a snapshot of the runtime properties and status of the VM" https://learn.microsoft.com/en-us/rest/api/compute/virtual-machines/instance-view?view=rest-compute-2025-04-01&tabs=HTTP Seems nice, but there's no diff in the json returned by `"instanceView"` and the default when `expander` is not set (referred to `model view` in some places on thier docs) 
```go
InstanceViewTypesInstanceView InstanceViewTypes = "instanceView"  
InstanceViewTypesUserData InstanceViewTypes = "userData"
```


IBM PR: #155368


155368: roachprod: implement GetVMSpecs for IBM r=DarrylWong,golgeek a=williamchoe3

Informs #146202

Implementation for previously unimplemented interface method GetVMSpecs() for IBM. This will allow for cluster information to be fetched in roachtest via FetchVMSpecs
* API: https://cloud.ibm.com/apidocs/resource-controller/resource-controller?code=go#get-resource-instance 

Also added comments to help contextualize some parts that I thought would be helpful while doing some debug.

See comment for e.g. vm specs


156504: sql: enable test tenants r=yuzefovich a=yuzefovich

**sql: remove some redundant kv.TestingIsRangeLookupRequest calls in tests**

These were redundant because we have GetRequests in all call sites.

**sql: enable test tenants**

Some tests have been adjusted to work with test tenants, others have specific issues to investigate further.

Fixes: #143114
Epic: CRDB-48945

156547: sql: add ColumnGeneratedAsIdentity as a new schema change element r=shghasemi a=shghasemi

Previously, GeneratedAsIdentity was part of the Column element.
Creating a new schema change element will allow the declarative schema changer
to add/remove/update such columns without dropping the Column element.

Epic: [CRDB-31283](https://cockroachlabs.atlassian.net/browse/CRDB-31283)

Part of: #142918

Release note: None

156940: util/taskset: add generic work distribution mechanism r=spilchen a=spilchen

Adds a new taskset package that provides task coordination for a number of workers. TaskSet hands out integer identifiers (TaskIDs) that workers claim and process, with the caller responsible for mapping TaskIDs to actual work (file indices, key ranges, batches, etc.).

Features round-robin initial distribution across workers and locality when getting a task ID. Not thread-safe; callers provide external synchronization.

This will be used by the new distributed merge pipeline.

Closes #156578

Epic: CRDB-48845
Release note: none

Co-authored by: `@jeffswenson`

156987: sql: remove now-stale growstack for internal executor r=yuzefovich a=yuzefovich

As of 4a5c503, we now grow stack unconditionally for all async tasks. Some time ago we added an ability to growstack in the async task of the internal executor when used in the LDR, and now that has become redundant, so we remove this special case.

Epic: None
Release note: None

Co-authored-by: William Choe <williamchoe3@gmail.com>
Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
Co-authored-by: Shadi Ghasemitaheri <shadi.ghasemitaheri@cockroachlabs.com>
Co-authored-by: Matt Spilchen <matt.spilchen@cockroachlabs.com>
@craig
Copy link
Copy Markdown
Contributor

craig bot commented Nov 6, 2025

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Nov 6, 2025

@craig craig bot merged commit 353e4b9 into cockroachdb:master Nov 6, 2025
33 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

o-AI-Review-Potential-Issue-Detected AI reviewer found potential issue. Never assign manually—auto-applied by GH action only. O-AI-Review-Real-Issue-Found AI reviewer found real issue v26.1.0-prerelease

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants