sql: replicating JSON empty array ordering found in Postgres#101260
Conversation
51b5d5d to
27563ce
Compare
yuzefovich
left a comment
There was a problem hiding this comment.
LGTM, I only have nits. I'll defer to Marcus for approval though.
Reviewed 6 of 6 files at r1, all commit messages.
Reviewable status:complete! 0 of 0 LGTMs obtained (waiting on @mgartner and @Shivs11)
-- commits line 18 at r1:
nit: I'd omit the release note since now the behavior matches Postgres (in other words, we are "fixing" difference from PG that was introduced in previous PRs, but those PRs haven't been a part of any release, so we don't need a "bug fix" release note).
pkg/util/encoding/encoding.go line 111 at r1 (raw file):
// Postgres currently has a special case (maybe a bug) where // the empty JSON Array sorts before all other JSON values.
nit: consider putting a link to the PG bug report here too.
pkg/util/encoding/encoding.go line 3477 at r1 (raw file):
// EncodeJSONArrayKeyMarker adds a JSON Array key encoding marker // to buf and returns the new buffer. func EncodeJSONArrayKeyMarker(buf []byte, dir Direction, length int64) []byte {
nit: perhaps s/length/arrayLength/g to be more explicit.
70a322a to
307477a
Compare
|
Before this is merged, we'd like to check the SQL 2023 spec which reportedly defines the ordering of JSON values. If the ordering in the spec does not follow this empty array inconsistency of Postgres, we're probably fine with sticking with the current ordering. If the spec defines an ordering completely different than Postgres, we'll have to reconsider our options. This needs to be done before v23.2 is shipped. |
307477a to
1e48064
Compare
mgartner
left a comment
There was a problem hiding this comment.
Reviewed 1 of 6 files at r1, 2 of 3 files at r3.
Reviewable status:complete! 0 of 0 LGTMs obtained (waiting on @Shivs11 and @yuzefovich)
Previously, yuzefovich (Yahor Yuzefovich) wrote…
nit: I'd omit the release note since now the behavior matches Postgres (in other words, we are "fixing" difference from PG that was introduced in previous PRs, but those PRs haven't been a part of any release, so we don't need a "bug fix" release note).
Done.
pkg/util/encoding/encoding.go line 111 at r1 (raw file):
Previously, yuzefovich (Yahor Yuzefovich) wrote…
nit: consider putting a link to the PG bug report here too.
Done.
pkg/util/encoding/encoding.go line 3477 at r1 (raw file):
Previously, yuzefovich (Yahor Yuzefovich) wrote…
nit: perhaps
s/length/arrayLength/gto be more explicit.
Done.
pkg/util/json/encoded.go line 613 at r3 (raw file):
return -1, nil } jDecoded, _ := j.decode()
Is it safe to ignore the error here? Probably not... I think we shouldn't decode the whole value either, just the marker to see if the array is empty.
1e48064 to
d39aca2
Compare
mgartner
left a comment
There was a problem hiding this comment.
@yuzefovich do you mind giving this another quick look? I did some refactoring.
Reviewed 2 of 6 files at r2, 1 of 3 files at r3, 2 of 2 files at r4, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @Shivs11)
pkg/util/json/encoded.go line 613 at r3 (raw file):
Previously, mgartner (Marcus Gartner) wrote…
Is it safe to ignore the error here? Probably not... I think we shouldn't decode the whole value either, just the marker to see if the array is empty.
Oh, we don't even need to decode in these cases. Cool. Done.
d39aca2 to
8696c52
Compare
yuzefovich
left a comment
There was a problem hiding this comment.
Also perhaps consider updating the tech note too.
Reviewed 2 of 6 files at r2, 2 of 3 files at r3, 2 of 2 files at r4, 1 of 1 files at r5, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (and 1 stale) (waiting on @Shivs11)
Previously, mgartner (Marcus Gartner) wrote…
Done.
nit: also the PR description needs an update.
pkg/util/json/json_test.go line 80 at r5 (raw file):
`false`, `true`, // In Postgres's sorting rules, the empty array comes before everything (even null),
nit: this comment should be updated.
mgartner
left a comment
There was a problem hiding this comment.
Also perhaps consider updating the tech note too.
Done.
Reviewed 1 of 1 files at r5, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (and 1 stale) (waiting on @Shivs11)
Previously, yuzefovich (Yahor Yuzefovich) wrote…
nit: also the PR description needs an update.
Done.
8696c52 to
207849e
Compare
Currently, cockroachdb#97928 and cockroachdb#99275 are responsible for laying out a lexicographical ordering for JSON columns to be forward indexable in nature. This ordering is based on the rules posted by Postgres and is in cockroachdb#99849. However, Postgres currently sorts the empty JSON array before any other JSON values. A Postgres bug report for this has been opened: https://www.postgresql.org/message-id/17873-826fdc8bbcace4f1%40postgresql.org This PR intends on replicating the Postgres behavior. Fixes cockroachdb#105668 Epic: CRDB-24501 Release note: None
207849e to
056c300
Compare
|
TFTR! bors r+ |
|
Build failed (retrying...): |
|
Build succeeded: |
Currently, #97928 and #99275 are responsible for laying out a
lexicographical ordering for JSON columns to be forward indexable in
nature. This ordering is based on the rules posted by Postgres and is
in #99849.
However, Postgres currently sorts the empty JSON array before any other
JSON values. A Postgres bug report for this has been opened:
https://www.postgresql.org/message-id/17873-826fdc8bbcace4f1%40postgresql.org
This PR intends on replicating the Postgres behavior.
Fixes #105668
Epic: CRDB-24501
Release note: None