Skip to content

sql: add support for MATERIALIZED VIEW with AS OF SYSTEM TIME#142259

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
rafiss:refresh-mat-view-aost
Mar 4, 2025
Merged

sql: add support for MATERIALIZED VIEW with AS OF SYSTEM TIME#142259
craig[bot] merged 1 commit intocockroachdb:masterfrom
rafiss:refresh-mat-view-aost

Conversation

@rafiss
Copy link
Copy Markdown
Collaborator

@rafiss rafiss commented Mar 4, 2025

Release note (sql change): Statements such as REFRESH MATERIALIZED VIEW
and CREATE MATERIALIZED VIEW can now be executed with an AS OF SYSTEM
TIME clause. These statements can still not be used in an explicit
transaction.

fixes #96391

@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented Mar 4, 2025

Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks.

It looks like your PR touches SQL parser code but doesn't add or edit parser tests. Please make sure you add or edit parser tests if you edit the parser.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@rafiss rafiss force-pushed the refresh-mat-view-aost branch from f8d0f39 to 0338396 Compare March 4, 2025 07:28
@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented Mar 4, 2025

It looks like your PR touches SQL parser code but doesn't add or edit parser tests. Please make sure you add or edit parser tests if you edit the parser.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@rafiss rafiss changed the title Refresh mat view aost sql: add support for MATERIALIZED VIEW with AS OF SYSTEM TIME Mar 4, 2025
@rafiss rafiss added backport-24.3.x Flags PRs that need to be backported to 24.3 backport-25.1.x labels Mar 4, 2025
@rafiss rafiss force-pushed the refresh-mat-view-aost branch from 0338396 to b1b5132 Compare March 4, 2025 11:02
@rafiss rafiss marked this pull request as ready for review March 4, 2025 15:59
@rafiss rafiss requested a review from a team as a code owner March 4, 2025 15:59
@rafiss rafiss requested a review from a team March 4, 2025 15:59
@rafiss rafiss requested a review from a team as a code owner March 4, 2025 15:59
@rafiss rafiss requested review from fqazi, spilchen and yuzefovich and removed request for a team and yuzefovich March 4, 2025 15:59
Copy link
Copy Markdown
Collaborator

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, just one suggestion for a test. :lgtm:

Reviewed 15 of 15 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @spilchen)


pkg/sql/logictest/testdata/logic_test/materialized_view line 329 at r1 (raw file):


statement ok
REFRESH MATERIALIZED VIEW mat_view_as_of AS OF SYSTEM TIME '$ts_after_drop_c'

Should we validate if before object creation?

Copy link
Copy Markdown
Contributor

@spilchen spilchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I just had a few questions.

Reviewed all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @rafiss)


pkg/sql/create_view.go line 230 at r1 (raw file):

					}

					if creationTime.Less(mostRecentModTime) {

Is there any concurrency issue here if one of the dependent tables is changed immediately after this check? I assume once this create view commits, then other schema changes on the dependent tables will pick up the dependency. But what if the schema change and create view are happening at the same time?


pkg/sql/opt/optbuilder/create_view.go line 117 at r1 (raw file):

			Syntax: cv,
			Schema: schID,
			// We need the view query to include user-defined types as a 3-part name to

nit: should this comment be removed? I think it was copied above, which seems like a better place for it.


pkg/sql/logictest/testdata/logic_test/materialized_view line 275 at r1 (raw file):


statement ok
CREATE MATERIALIZED VIEW mat_view_as_of AS SELECT a FROM tab_as_of AS OF SYSTEM TIME '$ts_before_insert_1'

If I understand it correctly, the ASOF time applies only during the refresh of the MQT. Is it valid to specify an ASOF time and request NO DATA? For instance, is this something that we allow:

CREATE MATERIALIZED VIEW v1 AS SELECT * FROM tab_as_of AS OF SYSTEM TIME '-5s' WITH NO DATA

Copy link
Copy Markdown
Collaborator Author

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @fqazi and @spilchen)


pkg/sql/create_view.go line 230 at r1 (raw file):

Previously, spilchen wrote…

Is there any concurrency issue here if one of the dependent tables is changed immediately after this check? I assume once this create view commits, then other schema changes on the dependent tables will pick up the dependency. But what if the schema change and create view are happening at the same time?

i believe we should be safe here. since this operation modifies the backrefs on the dependent tables, and a competing schema change would need to make its own modifications to the table descriptor, that means at commit time only one of them will be able to succeed.


pkg/sql/opt/optbuilder/create_view.go line 117 at r1 (raw file):

Previously, spilchen wrote…

nit: should this comment be removed? I think it was copied above, which seems like a better place for it.

done


pkg/sql/logictest/testdata/logic_test/materialized_view line 275 at r1 (raw file):

Previously, spilchen wrote…

If I understand it correctly, the ASOF time applies only during the refresh of the MQT. Is it valid to specify an ASOF time and request NO DATA? For instance, is this something that we allow:

CREATE MATERIALIZED VIEW v1 AS SELECT * FROM tab_as_of AS OF SYSTEM TIME '-5s' WITH NO DATA

good question. i added a test case for this, and although i don't see why anyone would want to use it, i don't see a reason to disallow it, as it still makes sense semantically.


pkg/sql/logictest/testdata/logic_test/materialized_view line 329 at r1 (raw file):

Previously, fqazi (Faizan Qazi) wrote…

Should we validate if before object creation?

i added a test.

@rafiss rafiss removed backport-24.3.x Flags PRs that need to be backported to 24.3 backport-25.1.x labels Mar 4, 2025
Release note (sql change): Statements such as REFRESH MATERIALIZED VIEW
and CREATE MATERIALIZED VIEW can now be executed with an AS OF SYSTEM
TIME clause. These statements can still not be used in an explicit
transaction.
Copy link
Copy Markdown
Collaborator Author

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @fqazi and @spilchen)


pkg/sql/create_view.go line 230 at r1 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

i believe we should be safe here. since this operation modifies the backrefs on the dependent tables, and a competing schema change would need to make its own modifications to the table descriptor, that means at commit time only one of them will be able to succeed.

i tested this manually and the correct thing happened. i started the CREATE MATERIALIZED VIEW, then made it sleep after this check. then i ran an ALTER TABLE on the dependent table in a different session. this ALTER TABLE succeeded.

when the CREATE MATERIALIZED VIEW unpaused, it resumed execution, but encountered a transaction retry error. the CREATE MATERIALIZED VIEW was automatically retried, and on this second attempt it failed this check so returned the correct error.

@rafiss rafiss force-pushed the refresh-mat-view-aost branch from b1b5132 to 2932e59 Compare March 4, 2025 22:37
@rafiss
Copy link
Copy Markdown
Collaborator Author

rafiss commented Mar 4, 2025

TFTRs!

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 4, 2025

@craig craig bot merged commit 1fdbeb2 into cockroachdb:master Mar 4, 2025
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

aost: allow materialized view refresh with follower reads

4 participants