Skip to content

vinyl: fix use-after-free of LSM tree in scheduler#10012

Merged
locker merged 1 commit intotarantool:masterfrom
locker:vy-scheduler-crash-fix
May 16, 2024
Merged

vinyl: fix use-after-free of LSM tree in scheduler#10012
locker merged 1 commit intotarantool:masterfrom
locker:vy-scheduler-crash-fix

Conversation

@locker
Copy link
Member

@locker locker commented May 15, 2024

Between picking an LSM tree from a heap and taking a reference to it in vy_task_new() there are a few places where the scheduler may yield:

  • in vy_worker_pool_get() to start a worker pool;
  • in vy_task_dump_new() to wait for a memory tree to be unpinned;
  • in vy_task_compaction_new() to commit an entry to the metadata log after splitting or coalescing a range.

If a concurrent fiber drops and deletes the LSM tree in the meanwhile, the scheduler will crash. To avoid that, let's take a reference to the LSM tree.

It's quite difficult to write a functional test for it without a bunch of ugly error injections so we rely on fuzzing tests.

Closes #9995

Between picking an LSM tree from a heap and taking a reference to it in
vy_task_new() there are a few places where the scheduler may yield:
 - in vy_worker_pool_get() to start a worker pool;
 - in vy_task_dump_new() to wait for a memory tree to be unpinned;
 - in vy_task_compaction_new() to commit an entry to the metadata log
   after splitting or coalescing a range.

If a concurrent fiber drops and deletes the LSM tree in the meanwhile,
the scheduler will crash. To avoid that, let's take a reference to
the LSM tree.

It's quite difficult to write a functional test for it without a bunch
of ugly error injections so we rely on fuzzing tests.

Closes tarantool#9995

NO_DOC=bug fix
NO_TEST=fuzzing
@locker locker requested a review from a team as a code owner May 15, 2024 12:21
@locker locker requested a review from nshy May 15, 2024 12:30
@coveralls
Copy link

Coverage Status

coverage: 87.071% (-0.03%) from 87.104%
when pulling 7dada41 on locker:vy-scheduler-crash-fix
into d36493c
on tarantool:master
.

@locker locker assigned locker and unassigned nshy May 15, 2024
@locker locker added the full-ci Enables all tests for a pull request label May 16, 2024
@locker locker merged commit 1c4605b into tarantool:master May 16, 2024
@locker locker deleted the vy-scheduler-crash-fix branch May 16, 2024 07:55
@locker
Copy link
Member Author

locker commented May 16, 2024

Cherry-picked to 2.11 and 3.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-ci Enables all tests for a pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tarantool has crashed in vy_mem_wait_pinned

4 participants