Skip to content

*: support vector index and adding/dropping vector index when doing syncTableSchema#9451

Merged
ti-chi-bot[bot] merged 5 commits intopingcap:feature/vector-indexfrom
Lloyd-Pottiger:add-index-ddl
Sep 25, 2024
Merged

*: support vector index and adding/dropping vector index when doing syncTableSchema#9451
ti-chi-bot[bot] merged 5 commits intopingcap:feature/vector-indexfrom
Lloyd-Pottiger:add-index-ddl

Conversation

@Lloyd-Pottiger
Copy link
Contributor

What problem does this PR solve?

Issue Number: ref #9032

Problem Summary:

What is changed and how it works?

  • Support the vector index from tableInfo.IndexInfo.
  • Support adding/dropping vector index when doing syncTableSchema.
*: support vector index and adding/dropping vector index when doing syncTableSchema

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

@ti-chi-bot ti-chi-bot bot added the release-note-none Denotes a PR that doesn't merit a release note. label Sep 20, 2024
@sre-bot
Copy link
Collaborator

sre-bot commented Sep 20, 2024

CLA assistant check
All committers have signed the CLA.

@ti-chi-bot ti-chi-bot bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Sep 20, 2024
@JaySon-Huang
Copy link
Contributor

/test pull-integration-test

zimulala and others added 3 commits September 23, 2024 10:30
…yncTableSchema. (pingcap#280)

Co-authored-by: JaySon <tshent@qq.com>
Co-authored-by: Lloyd-Pottiger <60744015+Lloyd-Pottiger@users.noreply.github.com>
Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com>
Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com>
Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com>
@Lloyd-Pottiger
Copy link
Contributor Author

/retest

1 similar comment
@Lloyd-Pottiger
Copy link
Contributor Author

/retest

Copy link
Contributor

@JaySon-Huang JaySon-Huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Sep 23, 2024

std::atomic_store(&original_table_header, std::make_shared<Block>(toEmptyBlock(original_table_columns)));

// release the lock because `applyLocalIndexChange` will try to acquire the lock
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to update applyLocalIndexChange in this comment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

ASSERT_EQ(expect_idx2.id, idx1.index_id);
ASSERT_EQ(100, idx1.column_id);
ASSERT_NE(nullptr, idx1.index_definition);
ASSERT_EQ(expect_idx2.vector_index->kind, idx1.index_definition->kind);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In open-source TiDB, we remove the field of kind. We use the indexinfo.Tp instead of this kind of information. link: https://github.com/pingcap/tidb/pull/55839/files#diff-645e83502d90c4de6385256f7499dcdc234ef8b6808559e19570eb2d05c25992R54

{
LocalIndexInfosPtr index_infos = std::make_shared<LocalIndexInfos>();
index_infos->reserve(table_info.columns.size() + table_info.index_infos.size());
for (const auto & col : table_info.columns)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We needn't use this logic, because we don't support adding a vector index with a column comment.

Copy link
Contributor

@JaySon-Huang JaySon-Huang Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The later PR merge the logic of function initLocalIndexInfos into generateLocalIndexInfos.

I think we can remove this for-loop after all PRs from the cse branch are merged? Or this could introduce unnecessary conflict when cherry-pick later PRs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it.

for (auto & iter : original_local_index_id_map)
{
// It means this index is create by column comments which we don't support drop index.
if (iter.first == DB::EmptyIndexID)
Copy link
Contributor

@zimulala zimulala Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto. And some tests need to update

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Co-authored-by: JaySon <tshent@qq.com>
Copy link
Contributor

@zimulala zimulala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Sep 25, 2024

@zimulala: adding LGTM is restricted to approvers and reviewers in OWNERS files.

Details

In response to this:

LGTM

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Sep 25, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JaySon-Huang, zanmato1984, zimulala

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [JaySon-Huang,zanmato1984]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Sep 25, 2024
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Sep 25, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-09-23 06:06:25.434971515 +0000 UTC m=+1459655.175395461: ☑️ agreed by JaySon-Huang.
  • 2024-09-25 01:59:28.531232691 +0000 UTC m=+1617638.271656629: ☑️ agreed by zanmato1984.

@ti-chi-bot ti-chi-bot bot merged commit 1a781c7 into pingcap:feature/vector-index Sep 25, 2024
@Lloyd-Pottiger Lloyd-Pottiger deleted the add-index-ddl branch September 27, 2024 09:19
@JaySon-Huang JaySon-Huang mentioned this pull request Sep 30, 2024
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants