Skip to content

sql: support a limited subset of correlated SRFs#26503

Merged
craig[bot] merged 4 commits intocockroachdb:masterfrom
knz:20180607-corr-srfs
Jun 12, 2018
Merged

sql: support a limited subset of correlated SRFs#26503
craig[bot] merged 4 commits intocockroachdb:masterfrom
knz:20180607-corr-srfs

Conversation

@knz
Copy link
Copy Markdown
Contributor

@knz knz commented Jun 7, 2018

Next-to-last step before solving #16971.

This patch extends the "project set" operator previously implemented
so that it can evaluate expressions using references to the rows of
the source. This makes it able to evaluate SRFs correlated with the
data source.

This patch then extends the transform of SRFs in render position
to use the new functionality. With this patch, a query like this now
runs properly: SELECT unnest(indkey) from pg_index

Although the execution logic is now able to run arbitrary correlated
SRFs, the planning for ROWS FROM is not yet adjusted to use this
functionality and therefore SRFs listed in ROWS FROM cannot be
correlated yet.

Release note (sql change): CockroachDB now can evaluate set-generating
functions with arguments that refer to the FROM clause. In particular
this makes it possible to use json_each(), json_object_keys() etc
over JSON columns.

cc @solongordon
cc @justinj

knz added 3 commits June 7, 2018 14:10
Release note: None
Release note: None
@knz knz requested review from a team, BramGruneir and RaduBerinde June 7, 2018 12:18
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@knz knz force-pushed the 20180607-corr-srfs branch 4 times, most recently from 7baf986 to 01bf327 Compare June 7, 2018 12:47
@knz knz force-pushed the 20180607-corr-srfs branch 2 times, most recently from 0e2ee99 to da68eba Compare June 7, 2018 13:10
@BramGruneir
Copy link
Copy Markdown
Member

:lgtm:

But probably a good idea to have another set of eyes on this too. Someone with a deeper history of planning.


Reviewed 1 of 1 files at r1, 1 of 1 files at r2, 1 of 1 files at r3, 6 of 6 files at r4.
Review status: all files reviewed at latest revision, all discussions resolved, all commit checks successful.


pkg/sql/opt_needed.go, line 114 at r4 (raw file):

				// needed. Remove it.
				n.exprs[i] = tree.DNull
			}

could this be an else if?


pkg/sql/logictest/testdata/logic_test/srfs, line 864 at r4 (raw file):

#          index_name,
#          ordinal_position
# ----

Are you missing an empty line at the end?


Comments from Reviewable

This patch extends the "project set" operator previously implemented
so that it can evaluate expressions using references to the rows of
the source. This makes it able to evaluate SRFs correlated with the
data source.

This patch then extends the transform of SRFs in render position
to use the new functionality. With this patch, a query like this now
runs properly: `SELECT unnest(indkey) from pg_index`

Although the execution logic is now able to run arbitrary correlated
SRFs, the planning for `ROWS FROM` is not yet adjusted to use this
functionality and therefore SRFs listed in `ROWS FROM` cannot be
correlated yet.

Release note (sql change): CockroachDB now can evaluate set-generating
functions with arguments that refer to the FROM clause. In particular
this makes it possible to use `json_each()`, `json_object_keys()` etc
over JSON columns.
@knz knz force-pushed the 20180607-corr-srfs branch from da68eba to e6488f9 Compare June 8, 2018 10:31
@knz
Copy link
Copy Markdown
Contributor Author

knz commented Jun 8, 2018

Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.


pkg/sql/opt_needed.go, line 114 at r4 (raw file):

Previously, BramGruneir (Bram Gruneir) wrote…

could this be an else if?

No but it can certainly be simplified. Done.


pkg/sql/logictest/testdata/logic_test/srfs, line 864 at r4 (raw file):

Previously, BramGruneir (Bram Gruneir) wrote…

Are you missing an empty line at the end?

Yep, thanks for noticing.


Comments from Reviewable

@knz knz requested a review from solongordon June 10, 2018 15:45
@solongordon
Copy link
Copy Markdown
Contributor

:lgtm: Great to see this functionality!


Reviewed 1 of 1 files at r1, 1 of 1 files at r3, 2 of 6 files at r4, 2 of 2 files at r5.
Review status: :shipit: complete! 0 of 0 LGTMs obtained (and 2 stale)


Comments from Reviewable

@knz
Copy link
Copy Markdown
Contributor Author

knz commented Jun 12, 2018

Thank you!

bors r+

craig bot pushed a commit that referenced this pull request Jun 12, 2018
26503: sql: support a limited subset of correlated SRFs r=knz a=knz

Next-to-last step before solving  #16971.

This patch extends the "project set" operator previously implemented
so that it can evaluate expressions using references to the rows of
the source. This makes it able to evaluate SRFs correlated with the
data source.

This patch then extends the transform of SRFs in render position
to use the new functionality. With this patch, a query like this now
runs properly: `SELECT unnest(indkey) from pg_index`

Although the execution logic is now able to run arbitrary correlated
SRFs, the planning for `ROWS FROM` is not yet adjusted to use this
functionality and therefore SRFs listed in `ROWS FROM` cannot be
correlated yet.

Release note (sql change): CockroachDB now can evaluate set-generating
functions with arguments that refer to the FROM clause. **In particular
this makes it possible to use `json_each()`, `json_object_keys()` etc
over JSON columns.**

cc @solongordon 
cc @justinj 

Co-authored-by: Raphael 'kena' Poss <knz@cockroachlabs.com>
@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jun 12, 2018

Build succeeded

@craig craig bot merged commit e6488f9 into cockroachdb:master Jun 12, 2018
@knz knz deleted the 20180607-corr-srfs branch June 12, 2018 06:39
@knz knz added the docs-todo label Jun 14, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants