storage: add local indexer scheduler#9383
storage: add local indexer scheduler#9383ti-chi-bot[bot] merged 8 commits intopingcap:feature/vector-indexfrom
Conversation
Maybe need some better design (or names), since after logical split the left and right segment will hold the same dmfile_id but different page_id. |
@breezewish rename to // The file id of the DMFile.
struct DMFileID
{
explicit DMFileID(PageIdU64 id_)
: id(id_)
{}
PageIdU64 id;
};
// The page id of the ColumnFileTiny.
struct ColumnFileTinyID
{
explicit ColumnFileTinyID(PageIdU64 id_)
: id(id_)
{}
PageIdU64 id;
};
using FileID = std::variant<DMFileID, ColumnFileTinyID>; |
|
|
||
| std::unique_lock lock(mutex); | ||
|
|
||
| const auto internal_task = std::make_shared<InternalTask>(InternalTask{ |
There was a problem hiding this comment.
It looks like internal_task will not be shared, how about using unique_ptr ?
There was a problem hiding this comment.
But internal_task should be copy-able
e1adea0 to
e1fac2f
Compare
Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com>
Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com>
Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com>
ab49f9a to
2a8f66f
Compare
Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com>
Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: CalvinNeo, JaySon-Huang The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
[LGTM Timeline notifier]Timeline:
|
|
/test pull-integration-test |
What problem does this PR solve?
Issue Number: ref #9032
Problem Summary:
What is changed and how it works?
Extract
LocalIndexerSchedulerfrom https://github.com/tidbcloud/tiflash-cse/pull/203 to reduce the difficulty of code review. Meanwhile,LocalIndexerScheduler::Task::dmfile_idsis changed toLocalIndexerScheduler::Task::file_ids, which means we use page ids to identify each task, so we can add task for ColumnFileTiny later.Check List
Tests
Side effects
Documentation
Release note