Skip to content

memtx: do not abort unrelated transactions on DDL#10833

Merged
locker merged 1 commit intotarantool:masterfrom
drewdzzz:memtx_mvcc_ddl_abort_related
Nov 28, 2024
Merged

memtx: do not abort unrelated transactions on DDL#10833
locker merged 1 commit intotarantool:masterfrom
drewdzzz:memtx_mvcc_ddl_abort_related

Conversation

@drewdzzz
Copy link
Contributor

@drewdzzz drewdzzz commented Nov 19, 2024

See the commits for details.

Closes #10377

@coveralls
Copy link

coveralls commented Nov 19, 2024

Coverage Status

coverage: 87.331% (+0.03%) from 87.304%
when pulling 6fefc7c on drewdzzz:memtx_mvcc_ddl_abort_related
into 92774dc
on tarantool:master
.

@drewdzzz drewdzzz marked this pull request as ready for review November 20, 2024 08:20
@drewdzzz drewdzzz requested a review from a team as a code owner November 20, 2024 08:20
Copy link
Member

@CuriousGeorgiy CuriousGeorgiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great that you found such a simple solution to this problem by reusing existing read/write sets! I'm loving it.

@CuriousGeorgiy CuriousGeorgiy removed their assignment Nov 22, 2024
@locker locker assigned drewdzzz and unassigned locker Nov 25, 2024
Currently we simply abort all concurrent transactions on any DDL. We've
already done a few workarounds for these problems - filter that prevents
DDL on temporary spaces from aborting fully remote transactions and vice
versa, and an engine flag `ENGINE_TXM_HANDLES_DDL` to prevent memtx DDL
from aborting vinyl transactions. The commit removes above-mentioned
temporary solutions and makes memtx abort only related transactions on DDL.

The new approach is to abort all writers and readers when a space is
invalidated (it happens on replace in space cache). To abort all
writers, we can iterate over all transactions and inspect their
statements. To abort all readers, MVCC trackers can be used. When MVCC
is disabled, we don't abort any transactions since memtx ones cannot yield.
Note that we swap indexes on DDL before the old space is invalidated, so
all index read gaps are actually moved to the new space. It's actually
incorrect but didn't break anything - we delete all transaction's
trackers when it is prepared or aborted, and we used to abort all
in-progress transactions on DDL. But with the new approach, we need
those gaps in the old space in order to find all space readers, so the
commit swaps read gaps back when moving space indexes.

A primary memtx difference is that it contains all system spaces, and
any schema change is a replace in a system space, hence, in memtx. So
the question is whether we should abort any active transactions on
schema changes that don't affect any space directly (for example, bump
of schema version on upgrade, replace in space `_schema`). The answer is
we shouldn't - all resources used by spaces are pinned, so we cannot
change, for example, a collation when it is used by a space, so let's
abort concurrent transactions only when a space is invalidated - it
happens on DDL operations directly affecting spaces (`space:format(...)`,
`space:create_index(...)` and so on).

Note that despite the commit populates `point_hole_item` with a new
member, its size remains the same - the object has a 8-byte alignment
and because of the `bool` in the end 7 bytes are wasted, so the new
member is put right to the end right before the boolean to use the space
used for alignment.

Closes #10377

NO_DOC=not documentable feature
@drewdzzz drewdzzz requested a review from locker November 26, 2024 11:59
@drewdzzz drewdzzz assigned locker and unassigned drewdzzz Nov 26, 2024
@locker locker assigned drewdzzz and unassigned locker Nov 26, 2024
@drewdzzz drewdzzz added the full-ci Enables all tests for a pull request label Nov 27, 2024
@drewdzzz
Copy link
Contributor Author

Decided to backport to 3.2 but not to 2.11.

@drewdzzz drewdzzz removed their assignment Nov 28, 2024
@locker locker merged commit 3ac35ea into tarantool:master Nov 28, 2024
@locker
Copy link
Member

locker commented Nov 28, 2024

Cherry-picked to 3.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-ci Enables all tests for a pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Do not abort all yielding memtx transactions when DDL is committed

5 participants