Skip to content

storage: delake TestPebbleMapClose#51810

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
jbowens:jackson/storage-diskmap-log
Jul 28, 2020
Merged

storage: delake TestPebbleMapClose#51810
craig[bot] merged 1 commit intocockroachdb:masterfrom
jbowens:jackson/storage-diskmap-log

Conversation

@jbowens
Copy link
Copy Markdown
Contributor

@jbowens jbowens commented Jul 22, 2020

The TestPebbleMapClose test failed in CI with a sstable remaining after
the clearing of the disk map (#51786). A delete-only only compaction
followed by a move compaction resulted in a single remaining sstable
containing the range tombstone.

Adding compactions to elide deletions in L6 in Pebble would allow us to
revert to the old test condition. See cockroachdb/pebble#838.

Also, update this test to print a hex representation of
the raw, prefixed sstable bounds as well for easier debugging.

Fixes #51786.

Release note: none

@jbowens jbowens requested review from itsbilal and petermattis July 22, 2020 23:59
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Contributor Author

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a second commit that deflakes the test without changes to Pebble by no longer requiring the LSM to be completely empty. Instead it just requires all the tables that existed before the disk map was closed no longer exist. It's a little less sensitive to Pebble compaction heuristic idiosyncrasies, but not as strong an assurance we deleted all previous data. Thoughts?

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @itsbilal and @petermattis)

Copy link
Copy Markdown
Collaborator

@petermattis petermattis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with this adjustment to the test passing heuristics, but curious about why the sstable containing only a range tombstone was not compacted out of existence? Was this sstable in L6? If yes, why was the range tombstone not dropped?

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @itsbilal and @petermattis)

@jbowens jbowens changed the title storage: add more detailed output to TestPebbleMapClose storage: delake TestPebbleMapClose Jul 23, 2020
The TestPebbleMapClose test failed in CI with a sstable remaining after
the clearing of the disk map (cockroachdb#51786). A delete-only only compaction
followed by a move compaction resulted in a single remaining sstable
containing the range tombstone.

Adding compactions to elide deletions in L6 in Pebble would allow us to
revert to the old test condition. See cockroachdb/pebble#838.

Also, update this test to print a hex representation of
the raw, prefixed sstable bounds as well for easier debugging.

Release note: none
@jbowens jbowens force-pushed the jackson/storage-diskmap-log branch from c3c0aab to 0c905e1 Compare July 27, 2020 19:07
Copy link
Copy Markdown
Contributor Author

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Squashed the commits and added the explanation for the flakiness to the commit message.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @itsbilal)

Copy link
Copy Markdown
Collaborator

@petermattis petermattis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @itsbilal)

@jbowens
Copy link
Copy Markdown
Contributor Author

jbowens commented Jul 28, 2020

TFTR!

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jul 28, 2020

Build succeeded:

@craig craig bot merged commit 3a1b423 into cockroachdb:master Jul 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

storage: TestPebbleMapClose failed

3 participants