Skip to content

[nexus] Remove SELECT FOR UPDATE, patch tests#6729

Merged
smklein merged 7 commits into
mainfrom
select_for_serializability
Oct 1, 2024
Merged

[nexus] Remove SELECT FOR UPDATE, patch tests#6729
smklein merged 7 commits into
mainfrom
select_for_serializability

Conversation

@smklein

@smklein smklein commented Sep 30, 2024

Copy link
Copy Markdown
Collaborator

Building off of:

This PR acknowledges that SELECT FOR UPDATE is a performance "optimization" in the situation where a blueprint's value is checked before performing a subsequent operation, as we do with setting network resources. It arguably could make performance worse in certain cases, as it locks out concurrent read operations from accessing the database.

This PR removes the usage of SELECT FOR UPDATE, and significantly overhauls the test_ensure_external_networking_bails_on_bad_target test to account for the concurrent control that CockroachDB may be performing with respect to operation re-ordering.

@smklein smklein changed the title remove SELECT FOR UPDATE [nexus] Remove SELECT FOR UPDATE, patch tests Oct 1, 2024
@smklein smklein marked this pull request as ready for review October 1, 2024 01:55
Comment on lines +2325 to +2326
#[tokio::test]
async fn test_ensure_external_networking_works_with_good_target() {

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I split this into a new test because it's really important that we actually perform a write after checking the target in the "bad case" below.

If we don't actually perform a write, then there is no circular dependency between transactions, and CockroachDB can logically re-order the "ensure resources" task before other operations that might write to the target, regardless of when it actually happened.

@jgallagher jgallagher left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The comment in the reworked test is excellent.

Comment thread nexus/db-queries/src/db/datastore/deployment.rs Outdated
@smklein smklein enabled auto-merge (squash) October 1, 2024 17:20

// Release `ensure_resources_task` to finish.
return_on_completion.store(true, Ordering::SeqCst);
// == WHAT ORDERING DO WE EXPECT?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😍

@andrewjstone andrewjstone left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome stuff @smklein

@smklein smklein merged commit 33c83aa into main Oct 1, 2024
@smklein smklein deleted the select_for_serializability branch October 1, 2024 19:00
hawkw pushed a commit that referenced this pull request Oct 2, 2024
Building off of:

-
#6229 (comment)
- #6694
- https://github.com/oxidecomputer/sqldance

This PR acknowledges that `SELECT FOR UPDATE` is a performance
"optimization" in the situation where a blueprint's value is checked
before performing a subsequent operation, as we do with setting network
resources. It arguably could make performance worse in certain cases, as
it locks out concurrent read operations from accessing the database.

This PR removes the usage of `SELECT FOR UPDATE`, and significantly
overhauls the `test_ensure_external_networking_bails_on_bad_target` test
to account for the concurrent control that CockroachDB may be performing
with respect to operation re-ordering.

---------

Co-authored-by: John Gallagher <john@oxidecomputer.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants