Skip to content

[nexus] Delete "inv_dataset" records from inventory collections#6632

Merged
smklein merged 4 commits into
mainfrom
delete-inv-dataset
Sep 23, 2024
Merged

[nexus] Delete "inv_dataset" records from inventory collections#6632
smklein merged 4 commits into
mainfrom
delete-inv-dataset

Conversation

@smklein

@smklein smklein commented Sep 23, 2024

Copy link
Copy Markdown
Collaborator

Fixes #6615

Does not patch existing systems (dogfood) which may have unnecessary inventory entries

@smklein smklein changed the title Delete inventory dataset [nexus] Delete "inv_dataset" records from inventory collections Sep 23, 2024
};

// Remove rows for datasets
let ndatasets =

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the portion of the PR that actually fixes the underlying issue

.await
.unwrap();
assert_eq!(0, count);
let count = schema::inv_dataset::dsl::inv_dataset

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the portion of the test where I should have looked for the inv_dataset being gone

/// NOTE: This test depends on the naming convention "inv_" prefix name
/// to identify pieces of the inventory.
#[tokio::test]
async fn test_inventory_deletion() {

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a new test I added, out of paranoia, to try to help catch future cases like this, if any "inv_..." tables are added in the future, and they are not deleted.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might fix #5305

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's not exactly a database snapshot, but querying for all present and future inv_ tables is an approximation

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah the ticket isn't so much about that specific implementation so much as having a test to catch this problem.

I can't think of why you'd need a snapshot. A possible risk is that another inventory collection is inserted by a background task in the meantime, but since this is a datastore test and not an app-level test, there's no Nexus, and so that should be impossible.

Do you think we should have a test for the representative collection that it has rows for all inv_* tables? I'm a little afraid this test won't catch this problem in the future because someone might forget to add it to that, too. (It wouldn't have caught this specific instance for that reason, right?)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm gonna merge this PR first, but yes, totally agreed, we should have a test for that. I'll file another issue.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

];
let zpools = vec![];
let datasets = vec![];
let zpools = vec![InventoryZpool {

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was another oversight, I really should have added both of these to the representative inventory.

In the future, if anything is added to this representative inventory, but not deleted, the test_inventory_deletion test should now help catch it.

@smklein smklein marked this pull request as ready for review September 23, 2024 17:10

@jgallagher jgallagher left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I'm on board with declaring #5305 fixed with this; it's by convention, but it's a pretty strong convention at this point.

@smklein smklein enabled auto-merge (squash) September 23, 2024 17:46
@smklein smklein merged commit 165ddc2 into main Sep 23, 2024
@smklein smklein deleted the delete-inv-dataset branch September 23, 2024 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Need to review entries in inv_dataset table

3 participants