Skip to content

sql: replicating JSON empty array ordering found in Postgres#101260

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
Shivs11:jsonb_forward_indexes_composite_testing_postgres_bug
Aug 8, 2023
Merged

sql: replicating JSON empty array ordering found in Postgres#101260
craig[bot] merged 1 commit intocockroachdb:masterfrom
Shivs11:jsonb_forward_indexes_composite_testing_postgres_bug

Conversation

@Shivs11
Copy link
Copy Markdown
Contributor

@Shivs11 Shivs11 commented Apr 11, 2023

Currently, #97928 and #99275 are responsible for laying out a
lexicographical ordering for JSON columns to be forward indexable in
nature. This ordering is based on the rules posted by Postgres and is
in #99849.

However, Postgres currently sorts the empty JSON array before any other
JSON values. A Postgres bug report for this has been opened:
https://www.postgresql.org/message-id/17873-826fdc8bbcace4f1%40postgresql.org

This PR intends on replicating the Postgres behavior.

Fixes #105668

Epic: CRDB-24501

Release note: None

@Shivs11 Shivs11 requested a review from a team as a code owner April 11, 2023 21:27
@Shivs11 Shivs11 requested a review from cucaroach April 11, 2023 21:27
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@Shivs11 Shivs11 requested review from mgartner and yuzefovich and removed request for cucaroach April 11, 2023 21:27
@Shivs11 Shivs11 force-pushed the jsonb_forward_indexes_composite_testing_postgres_bug branch 2 times, most recently from 51b5d5d to 27563ce Compare April 11, 2023 22:26
Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I only have nits. I'll defer to Marcus for approval though.

Reviewed 6 of 6 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @mgartner and @Shivs11)


-- commits line 18 at r1:
nit: I'd omit the release note since now the behavior matches Postgres (in other words, we are "fixing" difference from PG that was introduced in previous PRs, but those PRs haven't been a part of any release, so we don't need a "bug fix" release note).


pkg/util/encoding/encoding.go line 111 at r1 (raw file):

	// Postgres currently has a special case (maybe a bug) where
	// the empty JSON Array sorts before all other JSON values.

nit: consider putting a link to the PG bug report here too.


pkg/util/encoding/encoding.go line 3477 at r1 (raw file):

// EncodeJSONArrayKeyMarker adds a JSON Array key encoding marker
// to buf and returns the new buffer.
func EncodeJSONArrayKeyMarker(buf []byte, dir Direction, length int64) []byte {

nit: perhaps s/length/arrayLength/g to be more explicit.

@Shivs11 Shivs11 force-pushed the jsonb_forward_indexes_composite_testing_postgres_bug branch 2 times, most recently from 70a322a to 307477a Compare April 13, 2023 15:09
@mgartner
Copy link
Copy Markdown
Contributor

mgartner commented May 8, 2023

Before this is merged, we'd like to check the SQL 2023 spec which reportedly defines the ordering of JSON values. If the ordering in the spec does not follow this empty array inconsistency of Postgres, we're probably fine with sticking with the current ordering. If the spec defines an ordering completely different than Postgres, we'll have to reconsider our options. This needs to be done before v23.2 is shipped.

@mgartner mgartner force-pushed the jsonb_forward_indexes_composite_testing_postgres_bug branch from 307477a to 1e48064 Compare August 4, 2023 19:16
Copy link
Copy Markdown
Contributor

@mgartner mgartner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 6 files at r1, 2 of 3 files at r3.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @Shivs11 and @yuzefovich)


-- commits line 18 at r1:

Previously, yuzefovich (Yahor Yuzefovich) wrote…

nit: I'd omit the release note since now the behavior matches Postgres (in other words, we are "fixing" difference from PG that was introduced in previous PRs, but those PRs haven't been a part of any release, so we don't need a "bug fix" release note).

Done.


pkg/util/encoding/encoding.go line 111 at r1 (raw file):

Previously, yuzefovich (Yahor Yuzefovich) wrote…

nit: consider putting a link to the PG bug report here too.

Done.


pkg/util/encoding/encoding.go line 3477 at r1 (raw file):

Previously, yuzefovich (Yahor Yuzefovich) wrote…

nit: perhaps s/length/arrayLength/g to be more explicit.

Done.


pkg/util/json/encoded.go line 613 at r3 (raw file):

		return -1, nil
	}
	jDecoded, _ := j.decode()

Is it safe to ignore the error here? Probably not... I think we shouldn't decode the whole value either, just the marker to see if the array is empty.

@mgartner mgartner force-pushed the jsonb_forward_indexes_composite_testing_postgres_bug branch from 1e48064 to d39aca2 Compare August 4, 2023 20:00
Copy link
Copy Markdown
Contributor

@mgartner mgartner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@yuzefovich do you mind giving this another quick look? I did some refactoring.

Reviewed 2 of 6 files at r2, 1 of 3 files at r3, 2 of 2 files at r4, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @Shivs11)


pkg/util/json/encoded.go line 613 at r3 (raw file):

Previously, mgartner (Marcus Gartner) wrote…

Is it safe to ignore the error here? Probably not... I think we shouldn't decode the whole value either, just the marker to see if the array is empty.

Oh, we don't even need to decode in these cases. Cool. Done.

@mgartner mgartner force-pushed the jsonb_forward_indexes_composite_testing_postgres_bug branch from d39aca2 to 8696c52 Compare August 4, 2023 20:05
Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! :lgtm:

Also perhaps consider updating the tech note too.

Reviewed 2 of 6 files at r2, 2 of 3 files at r3, 2 of 2 files at r4, 1 of 1 files at r5, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (and 1 stale) (waiting on @Shivs11)


-- commits line 18 at r1:

Previously, mgartner (Marcus Gartner) wrote…

Done.

nit: also the PR description needs an update.


pkg/util/json/json_test.go line 80 at r5 (raw file):

		`false`,
		`true`,
		// In Postgres's sorting rules, the empty array comes before everything (even null),

nit: this comment should be updated.

Copy link
Copy Markdown
Contributor

@mgartner mgartner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also perhaps consider updating the tech note too.

Done.

Reviewed 1 of 1 files at r5, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (and 1 stale) (waiting on @Shivs11)


-- commits line 18 at r1:

Previously, yuzefovich (Yahor Yuzefovich) wrote…

nit: also the PR description needs an update.

Done.

@mgartner mgartner force-pushed the jsonb_forward_indexes_composite_testing_postgres_bug branch from 8696c52 to 207849e Compare August 4, 2023 20:36
Currently, cockroachdb#97928 and cockroachdb#99275 are responsible for laying out a
lexicographical ordering for JSON columns to be forward indexable in
nature. This ordering is based on the rules posted by Postgres and is
in cockroachdb#99849.

However, Postgres currently sorts the empty JSON array before any other
JSON values. A Postgres bug report for this has been opened:
https://www.postgresql.org/message-id/17873-826fdc8bbcace4f1%40postgresql.org

This PR intends on replicating the Postgres behavior.

Fixes cockroachdb#105668

Epic: CRDB-24501

Release note: None
@mgartner mgartner force-pushed the jsonb_forward_indexes_composite_testing_postgres_bug branch from 207849e to 056c300 Compare August 8, 2023 13:08
@mgartner
Copy link
Copy Markdown
Contributor

mgartner commented Aug 8, 2023

TFTR!

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 8, 2023

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 8, 2023

Build succeeded:

@craig craig bot merged commit 69bc4c6 into cockroachdb:master Aug 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sql: decide on JSON ordering before 23.2 release

4 participants