Skip to content

logictest: support async queries in logictests#79871

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
AlexTalks:logictest_async_queries
Apr 15, 2022
Merged

logictest: support async queries in logictests#79871
craig[bot] merged 1 commit intocockroachdb:masterfrom
AlexTalks:logictest_async_queries

Conversation

@AlexTalks
Copy link
Copy Markdown
Contributor

This change adds support for asynchronous queries in logictests, which
will execute the query in a separate goroutine, prior to awaiting and
validating the results when unblocked by any contending operations.
While previously logictests had supported asynchronous statements as
introduced in #79010, the query execution, which returns rows to be
validated, is now supported in asynchronous mode with this change.

This feature is supported with the following syntax:

query <typestring> async,<options...> namedQuery
<some query that may block during execution>
----
<expected query results>
...
<operations that would free the locks namedQuery is waiting on>
...
awaitquery namedQuery
<waits on namedQuery to complete and validate the results>

While this feature is not supported in conjunction with some query
options such as retry, or when placed in a repeat block, it does
support being run with the --rewrite flag. This change also fixes a
bug with asynchronous statements that could cause a race condition in
testing.

Release note: None

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@AlexTalks AlexTalks requested review from nvb and yuzefovich April 13, 2022 00:42
Copy link
Copy Markdown
Contributor Author

@AlexTalks AlexTalks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @nvanbenschoten and @yuzefovich)


pkg/sql/logictest/logic.go, line 3430 at r1 (raw file):

		go func() {
			startedChan <- struct{}{}
			res, err := db.Exec(execSQL)

@nvanbenschoten this was the cause of the race condition in the tests we were looking at recently, unfortunately. The goroutine was racing with other operations that would modify t.db.

Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Only nits from me, :lgtm:

Reviewed 2 of 2 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @AlexTalks and @nvanbenschoten)


-- commits, line 26 at r1:
nit: it sounds like the fix should be in a separate commit.


pkg/sql/logictest/logic.go, line 568 at r1 (raw file):

const (
	queryRewritePlaceholderPrefix = "__async_query_rewrite_placeholder"

nit: when defining a single const value we tend to not use the parenthesis and do it in a single line instead.


pkg/sql/logictest/logic.go, line 1071 at r1 (raw file):

	logicQuery

	// the channel on which to receive the query results, when completed.

nit: s/the/The/.

Copy link
Copy Markdown
Contributor Author

@AlexTalks AlexTalks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @nvanbenschoten and @yuzefovich)


-- commits, line 26 at r1:

Previously, yuzefovich (Yahor Yuzefovich) wrote…

nit: it sounds like the fix should be in a separate commit.

I'll fix up the text here, but I put the fix in a separate PR already actually: #79925


pkg/sql/logictest/logic.go, line 568 at r1 (raw file):

Previously, yuzefovich (Yahor Yuzefovich) wrote…

nit: when defining a single const value we tend to not use the parenthesis and do it in a single line instead.

OK 👍


pkg/sql/logictest/logic.go, line 1071 at r1 (raw file):

Previously, yuzefovich (Yahor Yuzefovich) wrote…

nit: s/the/The/.

OK 👍

@AlexTalks AlexTalks force-pushed the logictest_async_queries branch from 6a60d91 to 0a784a8 Compare April 14, 2022 20:49
@AlexTalks AlexTalks force-pushed the logictest_async_queries branch from 0a784a8 to 08221ef Compare April 14, 2022 22:58
@AlexTalks
Copy link
Copy Markdown
Contributor Author

bors r+

craig bot pushed a commit that referenced this pull request Apr 14, 2022
79871: logictest: support async queries in logictests r=AlexTalks a=AlexTalks

This change adds support for asynchronous queries in logictests, which
will execute the query in a separate goroutine, prior to awaiting and
validating the results when unblocked by any contending operations.
While previously logictests had supported asynchronous statements as
introduced in #79010, the `query` execution, which returns rows to be
validated, is now supported in asynchronous mode with this change.

This feature is supported with the following syntax:
```
query <typestring> async,<options...> namedQuery
<some query that may block during execution>
----
<expected query results>
...
<operations that would free the locks namedQuery is waiting on>
...
awaitquery namedQuery
<waits on namedQuery to complete and validate the results>
```

While this feature is not supported in conjunction with some query
options such as `retry`, or when placed in a `repeat` block, it does
support being run with the `--rewrite` flag. This change also fixes a
bug with asynchronous statements that could cause a race condition in
testing.

Release note: None

Co-authored-by: Alex Sarkesian <sarkesian@cockroachlabs.com>
@craig
Copy link
Copy Markdown
Contributor

craig bot commented Apr 14, 2022

Build failed:

This change adds support for asynchronous queries in logictests, which
will execute the query in a separate goroutine, prior to awaiting and
validating the results when unblocked by any contending operations.
While previously logictests had supported asynchronous statements as
introduced in cockroachdb#79010, the `query` execution, which returns rows to be
validated, is now supported in asynchronous mode with this change.

This feature is supported with the following syntax:
```
query <typestring> async,<options...> namedQuery
<some query that may block during execution>
----
<expected query results>
...
<operations that would free the locks namedQuery is waiting on>
...
awaitquery namedQuery
<waits on namedQuery to complete and validate the results>
```

While this feature is not supported in conjunction with some query
options such as `retry`, or when placed in a `repeat` block, it does
support being run with the `--rewrite` flag.

Release note: None
@AlexTalks AlexTalks force-pushed the logictest_async_queries branch from 08221ef to 1613edb Compare April 15, 2022 20:16
@AlexTalks
Copy link
Copy Markdown
Contributor Author

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Apr 15, 2022

Build succeeded:

@craig craig bot merged commit 5f5f1f3 into cockroachdb:master Apr 15, 2022
@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented Apr 15, 2022

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from 1613edb to blathers/backport-release-22.1-79871: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 22.1.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants