Skip to content

pgwire: add encoding for JSON arguments#88379

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
rafiss:json-encoding
Sep 23, 2022
Merged

pgwire: add encoding for JSON arguments#88379
craig[bot] merged 1 commit intocockroachdb:masterfrom
rafiss:json-encoding

Conversation

@rafiss
Copy link
Copy Markdown
Collaborator

@rafiss rafiss commented Sep 21, 2022 •
edited
Loading

fixes #88355

This adds pgwire decoding support for JSON (oid=114). This does not add
oid=114 to the OidToType map in types.go since that introduces many
other assumptions about the type; namely that the type can be used as
the type of a column descriptor. That change would mean that existing
tables that were defined with JSON would be inconsistent with new tables
defined with a JSON column.

Release note (sql change): The pgwire protocol implementation can
now accept arguments of the JSON type (oid=114). Previously, it could
only accept JSONB (oid=3802). Internally, JSON and JSONB values are
still identical, so this change only affects how the values are received
over the wire protocol.

@rafiss rafiss requested review from a team, ZhouXing19 and otan September 21, 2022 19:09
@rafiss rafiss requested a review from a team as a code owner September 21, 2022 19:09
@rafiss rafiss requested a review from a team September 21, 2022 19:09
@cockroach-teamcity
Copy link
Copy Markdown
Member

cockroach-teamcity commented Sep 21, 2022

This change is Reviewable

otan
otan approved these changes Sep 21, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@otan otan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

me gusta this solution vs implementing the whole type!

@otan
Copy link
Copy Markdown
Contributor

otan commented Sep 21, 2022

hmm, i did miss the test failures though:

[TestCleanupDoesNotDeleteParentsWithChildObjects/clean-up-database-with-schema] === RUN   TestCleanupDoesNotDeleteParentsWithChildObjects/clean-up-database-with-schema
panic: oid 0 couldn't be mapped to array oid [recovered]
	panic: oid 0 couldn't be mapped to array oid

goroutine 22090 [running]:
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).recover(0xc0046863f0, {0x6012450, 0xc008552690})
	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:248 +0xa5
panic({0x47ebd80, 0xc000af9950})
	GOROOT/src/runtime/panic.go:884 +0x212
github.com/cockroachdb/cockroach/pkg/sql/types.CalcArrayOid(0x87bf7a0?)
	github.com/cockroachdb/cockroach/pkg/sql/types/pkg/sql/types/oid.go:231 +0x10e
github.com/cockroachdb/cockroach/pkg/sql.addPGTypeRow({{0x6042c10?, 0xc008484000?}}, {0x606eb88?, 0xc007da75e0}, {0x606f508?, 0x8b87820}, 0x87bf7a0, 0xc001ecd380)
	github.com/cockroachdb/cockroach/pkg/sql/pg_catalog.go:3035 +0x3b5
github.com/cockroachdb/cockroach/pkg/sql.glob..func235.1({0x6099b80, 0xc0056b2d80})
	github.com/cockroachdb/cockroach/pkg/sql/pg_catalog.go:3120 +0x13e
github.com/cockroachdb/cockroach/pkg/sql.forEachDatabaseDesc({0x6012450, 0xc008552690}, 0xc004cc3dd0, {0x6099b80?, 0xc0056b2d80?}, 0x0, 0xc00544fd50)
	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2349 +0x232
github.com/cockroachdb/cockroach/pkg/sql.glob..func235({0x6012450, 0xc008552690}, 0xc004cc3dd0, {0x6099b80, 0xc0056b2d80}, 0xc001ecd380)
	github.com/cockroachdb/cockroach/pkg/sql/pg_catalog.go:3114 +0x105
github.com/cockroachdb/cockroach/pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1({0x6012450, 0xc008552690}, {0x5fe28c0?, 0xc007da7580})
	github.com/cockroachdb/cockroach/pkg/sql/virtual_schema.go:572 +0x156
github.com/cockroachdb/cockroach/pkg/sql.setupGenerator.func3({0x6012450, 0xc008552690})
	github.com/cockroachdb/cockroach/pkg/sql/virtual_table.go:127 +0x114
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:489 +0x146
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:480 +0x43b

that seems ... undesirable. let me know if this turns out to have a surprise change and i'll review it again.

by the looks of it you "just" need an entry in oidToArrayOid

@rafiss rafiss force-pushed the json-encoding branch 3 times, most recently from b660d6f to 3c76d09 Compare September 22, 2022 20:02
@rafiss rafiss requested a review from otan September 22, 2022 20:02
@rafiss
Copy link
Copy Markdown
Collaborator Author

rafiss commented Sep 22, 2022

@otan could you give it another review? i ended up taking the type out of the OidToType map since that would be too invasive. there are some smaller special cases now when decoding in pgwire. (see commit message too.)

@rafiss
pgwire: add encoding for JSON arguments
1fe4c77 
This adds pgwire decoding support for JSON (oid=114). This does not add
oid=114 to the OidToType map in types.go since that introduces many
other assumptions about the type; namely that the type can be used as
the type of a column descriptor. That change would mean that existing
tables that were defined with JSON would be inconsistent with new tables
defined with a JSON column.

Release note (sql change): The pgwire protocol implementation can
now accept arguments of the JSON type (oid=114). Previously, it could
only accept JSONB (oid=3802). Internally, JSON and JSONB values are
still identical, so this change only affects how the values are received
over the wire protocol.
ZhouXing19
ZhouXing19 approved these changes Sep 23, 2022
View reviewed changes
Copy link
Copy Markdown
Collaborator

@ZhouXing19 ZhouXing19 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@rafiss
Copy link
Copy Markdown
Collaborator Author

rafiss commented Sep 23, 2022

tftr!

bors r=ZhouXing19

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Sep 23, 2022

Build succeeded:

@craig craig bot merged commit 14f11eb into cockroachdb:master Sep 23, 2022
@blathers-crl blathers-crl bot mentioned this pull request Sep 23, 2022
Merged
@cockroach-teamcity cockroach-teamcity mentioned this pull request Sep 23, 2022
Closed
@rafiss rafiss deleted the json-encoding branch September 23, 2022 21:15
@ZhouXing19 ZhouXing19 mentioned this pull request Jan 5, 2023
Merged
ZhouXing19 added a commit to ZhouXing19/cockroach that referenced this pull request Jan 5, 2023
@ZhouXing19
sql/pgwire: support for decoding JSON[] OID by aliasing to JSONB[] OID
a208ef6 
fixes cockroachdb#90839

Previously, with the parameter type oid (199) for JSON[], the pgwire parse message
will fail to be executed with a `unknown oid type: 199`. We now aliasing it
to the one for `JSONB[]`.

This commit follows the changes in cockroachdb#88379.

Release note (sql change): The pgwire protocol implementation can
now accept arguments of the JSON[] type (oid=199). Previously, it could
only accept JSONB[] (oid=3804). Internally, JSON[] and JSONB[] values are
still identical, so this change only affects how the values are received
over the wire protocol.
ZhouXing19 added a commit to ZhouXing19/cockroach that referenced this pull request Jan 6, 2023
@ZhouXing19
sql/pgwire: support for decoding JSON[] OID by aliasing to JSONB[] OID
01ac0df 
fixes cockroachdb#90839

Previously, with the parameter type oid (199) for JSON[], the pgwire parse message
will fail to be executed with a `unknown oid type: 199`. We now aliasing it
to the one for `JSONB[]`.

This commit follows the changes in cockroachdb#88379.

Release note (sql change): The pgwire protocol implementation can
now accept arguments of the JSON[] type (oid=199). Previously, it could
only accept JSONB[] (oid=3804). Internally, JSON[] and JSONB[] values are
still identical, so this change only affects how the values are received
over the wire protocol.
ZhouXing19 added a commit to ZhouXing19/cockroach that referenced this pull request Jan 6, 2023
@ZhouXing19
sql/pgwire: support for decoding JSON[] OID by aliasing to JSONB[] OID
5c603ca 
fixes cockroachdb#90839

Previously, with the parameter type oid (199) for JSON[], the pgwire parse message
will fail to be executed with a `unknown oid type: 199`. We now aliasing it
to the one for `JSONB[]`.

This commit follows the changes in cockroachdb#88379.

Release note (sql change): The pgwire protocol implementation can
now accept arguments of the JSON[] type (oid=199). Previously, it could
only accept JSONB[] (oid=3804). Internally, JSON[] and JSONB[] values are
still identical, so this change only affects how the values are received
over the wire protocol.
ZhouXing19 added a commit to ZhouXing19/cockroach that referenced this pull request Jan 18, 2023
@ZhouXing19
sql/pgwire: support for decoding JSON[] OID by aliasing to JSONB[] OID
7a0ac0c 
fixes cockroachdb#90839

Previously, with the parameter type oid (199) for JSON[], the pgwire parse message
will fail to be executed with a `unknown oid type: 199`. We now aliasing it
to the one for `JSONB[]`.

This commit follows the changes in cockroachdb#88379.

Release note (sql change): The pgwire protocol implementation can
now accept arguments of the JSON[] type (oid=199). Previously, it could
only accept JSONB[] (oid=3804). Internally, JSON[] and JSONB[] values are
still identical, so this change only affects how the values are received
over the wire protocol.
ZhouXing19 added a commit to ZhouXing19/cockroach that referenced this pull request Jan 19, 2023
@ZhouXing19
sql/pgwire: support for decoding JSON[] OID by aliasing to JSONB[] OID
ce51054 
fixes cockroachdb#90839

Previously, with the parameter type oid (199) for JSON[], the pgwire parse message
will fail to be executed with a `unknown oid type: 199`. We now aliasing it
to the one for `JSONB[]`.

This commit follows the changes in cockroachdb#88379.

Release note (sql change): The pgwire protocol implementation can
now accept arguments of the JSON[] type (oid=199). Previously, it could
only accept JSONB[] (oid=3804). Internally, JSON[] and JSONB[] values are
still identical, so this change only affects how the values are received
over the wire protocol.
craig bot pushed a commit that referenced this pull request Jan 19, 2023
@ZhouXing19
Merge #94705
0bc78ed 
94705: sql/pgwire: support for decoding JSON[] OID by aliasing to JSONB[] OID r=ZhouXing19 a=ZhouXing19

Previously, with the parameter type oid (199) for JSON[], the pgwire parse message
will fail to be executed with a `unknown oid type: 199`. We now aliasing it
to the one for `JSONB[]`.

This PR also includes a test revealing #95434.

This commit follows the changes in #88379.

fixes #90839

Release note (sql change): The pgwire protocol implementation can
now accept arguments of the JSON[] type (oid=199). Previously, it could
only accept JSONB[] (oid=3804). Internally, JSON[] and JSONB[] values are
still identical, so this change only affects how the values are received
over the wire protocol.

Co-authored-by: Jane Xing <zhouxing@uchicago.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ZhouXing19 ZhouXing19 ZhouXing19 approved these changes

@otan otan Awaiting requested review from otan

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

"unknown oid type: 114" when sending JSON data in wire format

4 participants

@rafiss @cockroach-teamcity @otan @ZhouXing19