Skip to content

memtx: use index definition's key_def for point hole comparisons#11388

Merged
sergepetrenko merged 8 commits intotarantool:masterfrom
CuriousGeorgiy:mvcc-bitwise-key-comparison
May 5, 2025
Merged

memtx: use index definition's key_def for point hole comparisons#11388
sergepetrenko merged 8 commits intotarantool:masterfrom
CuriousGeorgiy:mvcc-bitwise-key-comparison

Conversation

@CuriousGeorgiy
Copy link
Member

This patchset fixes the incorrect assumption that point hole equivalence classes are defined by
their keys' binary (MsgPack) representation.

Closes #10159
Closes #11292

@CuriousGeorgiy CuriousGeorgiy requested a review from a team as a code owner April 12, 2025 17:47
@CuriousGeorgiy CuriousGeorgiy force-pushed the mvcc-bitwise-key-comparison branch from 533e59a to 159a713 Compare April 13, 2025 13:07
@CuriousGeorgiy CuriousGeorgiy added the asan-ci Enables asan build tests for a pull request label Apr 13, 2025
@coveralls
Copy link

coveralls commented Apr 13, 2025

Coverage Status

coverage: 87.499% (+0.01%) from 87.486%
when pulling 298599e on CuriousGeorgiy:mvcc-bitwise-key-comparison
into 28cd485
on tarantool:master
.

@CuriousGeorgiy CuriousGeorgiy force-pushed the mvcc-bitwise-key-comparison branch 2 times, most recently from 60f3c7e to fd37217 Compare April 13, 2025 16:17
Copy link
Contributor

@mkostoevr mkostoevr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to see the MVCC subsystem getting cleaner.

Copy link
Contributor

@drewdzzz drewdzzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch!
We've fixed this old minor problem finally 😄
Also, it's nice to see that MVCC code becomes a little more tidy.

@drewdzzz drewdzzz assigned CuriousGeorgiy and unassigned mkostoevr and drewdzzz Apr 14, 2025
MVCC storages are supposed to be empty by the time MVCC is freed — let's
add assertions to check this assumption.

Needed for tarantool#10159
Needed for tarantool#11292

NO_CHANGELOG=<refactoring>
NO_DOC=<refactoring>
NO_TEST=<refactoring>
We combine the index and tuple hash for point holes in a non-trivial way in
more than one place — let's encapsulate the combination logic in a separate
function for future changes.

Needed for tarantool#10159
Needed for tarantool#11292

NO_CHANGELOG=<refactoring>
NO_DOC=<refactoring>
NO_TEST=<refactoring>
It is safe to remove an object from its list, even if the list is empty —
let's always remove point holes from their rings to simplify the code. We
will use this to further separate the point hole storage deletion from the
point hole item deletion.

Needed for tarantool#10159
Needed for tarantool#11292

NO_CHANGELOG=<refactoring>
NO_DOC=<refactoring>
NO_TEST=<refactoring>
We delete point item separately in more than one place — let's encapsulate
this logic in a separate function for further changes.

Needed for tarantool#10159
Needed for tarantool#11292

NO_CHANGELOG=<refactoring>
NO_DOC=<refactoring>
NO_TEST=<refactoring>
Globally unique index identifiers are redundant, since indexes can be
uniquely identified by pointers — let's use the pointer to the index for
this purpose.

Needed for tarantool#10159
Needed for tarantool#11292

NO_CHANGELOG=<refactoring>
NO_DOC=<refactoring>
NO_TEST=<refactoring>
Currently, we assume that point hole equivalence classes are defined by
their keys' binary (MsgPack) representation, and use `memcmp` for
comparison. However, this is not true in the case of numeric key
definitions and collations.

To fix this, let's use the corresponding index's key definition for
comparison. Storing the key length then becomes redundant, so let's drop
it.

Closes tarantool#10159
Closes tarantool#11292

NO_DOC=<bugfix>
The `space_id` field has become redundant after we started saving the
`index` to point holes — let's drop it to simplify the code.

Folows-up tarantool#10159
Folows-up tarantool#11292

NO_CHANGELOG=<refactoring>
NO_DOC=<refactoring>
NO_TEST=<refactoring>
@CuriousGeorgiy CuriousGeorgiy force-pushed the mvcc-bitwise-key-comparison branch from fd37217 to 70a5fc4 Compare April 14, 2025 14:20
@CuriousGeorgiy
Copy link
Member Author

CuriousGeorgiy commented Apr 14, 2025

@mkostoevr @drewdzzz I added a small follow-up patch that reuses the space_id from index in point holes and drops the space_id field from them to further simplify the code — I hope you don't mind.

@CuriousGeorgiy CuriousGeorgiy added full-ci Enables all tests for a pull request backport/2.11 Automatically create a 2.11 backport PR backport/3.2 Automatically create a 3.2 backport PR backport/3.3 Automatically create a 3.3 backport PR and removed asan-ci Enables asan build tests for a pull request full-ci Enables all tests for a pull request labels Apr 14, 2025
@drewdzzz
Copy link
Contributor

@CuriousGeorgiy

I added a small follow-up patch that reuses the space_id from index in point holes and drops the space_id field from them to further simplify the code — I hope you don't mind.

Please, move uint32_t hash field to the end of the structure right before bool is_head in order to decrease size of the structure. Let's do it right in the same commit with drop of space_id.

@CuriousGeorgiy CuriousGeorgiy force-pushed the mvcc-bitwise-key-comparison branch from 6da1e5d to 254b56a Compare April 14, 2025 16:54
@CuriousGeorgiy CuriousGeorgiy added the full-ci Enables all tests for a pull request label Apr 14, 2025
Folows-up tarantool#10159
Folows-up tarantool#11292

NO_CHANGELOG=<optimization>
NO_DOC=<optimization>
@CuriousGeorgiy CuriousGeorgiy force-pushed the mvcc-bitwise-key-comparison branch from 254b56a to 298599e Compare April 15, 2025 07:55
@sergepetrenko sergepetrenko merged commit 76b114e into tarantool:master May 5, 2025
44 checks passed
@TarantoolBot
Copy link
Collaborator

Backport failed for release/2.11, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release/2.11
git worktree add -d .worktree/backport/release/2.11/11388 origin/release/2.11
cd .worktree/backport/release/2.11/11388
git switch --create backport/release/2.11/11388
git cherry-pick -x 0a02acb72c9dd4b6eb80cea881f346b71dad2323 b6cababf1883b05e77e894212f9234c965007cd9 171412abf7fbc93abe2bfde141bf21a421948c1f 674d75e4aa82d00b114404bdea9055182fcc0deb 19ac49a76fdb9b4309539817689fc401d35034e4 b2fc50179c6ca09288e116a5760b5791a38e026a f76d3f901ab177b9778aac2070ca0600bfadfec4 76b114e9ad9ea04868cc05235dde7075cc669a54

@TarantoolBot
Copy link
Collaborator

Backport failed for release/3.2, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release/3.2
git worktree add -d .worktree/backport/release/3.2/11388 origin/release/3.2
cd .worktree/backport/release/3.2/11388
git switch --create backport/release/3.2/11388
git cherry-pick -x 0a02acb72c9dd4b6eb80cea881f346b71dad2323 b6cababf1883b05e77e894212f9234c965007cd9 171412abf7fbc93abe2bfde141bf21a421948c1f 674d75e4aa82d00b114404bdea9055182fcc0deb 19ac49a76fdb9b4309539817689fc401d35034e4 b2fc50179c6ca09288e116a5760b5791a38e026a f76d3f901ab177b9778aac2070ca0600bfadfec4 76b114e9ad9ea04868cc05235dde7075cc669a54

@TarantoolBot
Copy link
Collaborator

Backport failed for release/3.3, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release/3.3
git worktree add -d .worktree/backport/release/3.3/11388 origin/release/3.3
cd .worktree/backport/release/3.3/11388
git switch --create backport/release/3.3/11388
git cherry-pick -x 0a02acb72c9dd4b6eb80cea881f346b71dad2323 b6cababf1883b05e77e894212f9234c965007cd9 171412abf7fbc93abe2bfde141bf21a421948c1f 674d75e4aa82d00b114404bdea9055182fcc0deb 19ac49a76fdb9b4309539817689fc401d35034e4 b2fc50179c6ca09288e116a5760b5791a38e026a f76d3f901ab177b9778aac2070ca0600bfadfec4 76b114e9ad9ea04868cc05235dde7075cc669a54

@sergepetrenko sergepetrenko added the backport/3.4 Automatically create a 3.4 backport PR label May 6, 2025
@sergepetrenko
Copy link
Collaborator

@CuriousGeorgiy, please, prepare the backports to all the relevant branches.

@CuriousGeorgiy
Copy link
Member Author

/backport 3.4

@TarantoolBot
Copy link
Collaborator

Successfully created backport PR for release/3.4:

@TarantoolBot
Copy link
Collaborator

Backport summary

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport/2.11 Automatically create a 2.11 backport PR backport/3.2 Automatically create a 3.2 backport PR backport/3.3 Automatically create a 3.3 backport PR backport/3.4 Automatically create a 3.4 backport PR full-ci Enables all tests for a pull request

Projects

None yet

7 participants