Merged
Conversation
ref #9032 Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> Co-authored-by: JaySon-Huang <tshent@qq.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref #9032 Co-authored-by: JaySon-Huang <tshent@qq.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref #9032 storage: Use mmap to view vector index Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> Co-authored-by: JaySon-Huang <tshent@qq.com>
ref #9032 storage: Add vector search metrics Signed-off-by: Wish <breezewish@outlook.com> Co-authored-by: Wenxuan <breezewish@outlook.com>
ref #9032 *: use SimSIMD for vectors Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com>
ref #9032 storage: Add system.dt_local_indexes Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com>
ref #9032 DMFile: Support modify DMFile meta --------- Signed-off-by: Wish <breezewish@outlook.com> Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> Co-authored-by: Wenxuan <breezewish@outlook.com> Co-authored-by: JaySon <tshent@qq.com>
ref #9032 storage: Force evict when downloading vector index files Signed-off-by: Wish <breezewish@outlook.com> Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> Co-authored-by: Wenxuan <breezewish@outlook.com>
ref #9032 storage: add local indexer scheduler Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref #9032 storage: Support adding vector index in background Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com>
close #9032 storage: Abort vector index building as soon as possible Signed-off-by: Wish <breezewish@outlook.com> Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> Co-authored-by: Wenxuan <breezewish@outlook.com> Co-authored-by: Lloyd-Pottiger <60744015+Lloyd-Pottiger@users.noreply.github.com>
ref #9032 ddl: Support parsing VectorIndex defined in IndexInfo Co-authored-by: JaySon <tshent@qq.com>
ref #9032 storage: support the HTTP API of sync table schema Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> Co-authored-by: Lynn <zimu_xia@126.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref #9032 storage: cache PK column in memory Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref #9032 ddl: Adapt with the latest vector index def
ref #9032 Storage: Support multiple vec indexes on the same column Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> Co-authored-by: JaySon <tshent@qq.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
…is dropped (#9475) ### What problem does this PR solve? Issue Number: ref #9032 Problem Summary: ### What is changed and how it works? Pick tidbcloud/tiflash-cse#283 and tidbcloud/tiflash-cse#300 * Unify the logic of `generateLocalIndexInfos` and `initLocalIndexInfos` * Print 1 logging for the vector index added/dropped/existing in one table instead. This can avoid the flood of logging when tiflash restart with lots of table with vector index defined * Support drop the vector index defined on ColumnInfo after the Column has been dropped in TiDB * Add more ut in the DeltaMergeStore read level * vector search fallback when top_k = max uint32 ```commit-message ``` ### Check List Tests <!-- At least one of them must be included. --> - [ ] Unit test - [ ] Integration test - [ ] Manual test (add detailed scripts or steps below) - [ ] No code Side effects - [ ] Performance regression: Consumes more CPU - [ ] Performance regression: Consumes more Memory - [ ] Breaking backward compatibility Documentation - [ ] Affects user behaviors - [ ] Contains syntax changes - [ ] Contains variable changes - [ ] Contains experimental features - [ ] Changes MySQL compatibility ### Release note <!-- bugfix or new feature needs a release note --> ```release-note None ``` --------- Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> Co-authored-by: JaySon <tshent@qq.com> Co-authored-by: jinhelin <linjinhe33@gmail.com>
Contributor
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Contributor
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
) close #9485 vector: Fix ColumnArray does not work well with CHBlockChunkCodec Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Storage: Add error message when fail to build local index
storage: remove vector_index in column level
Contributor
There was a problem hiding this comment.
temporary disable clang-tidy for this PR because too many file changes and it takes too long
zanmato1984
approved these changes
Sep 30, 2024
Contributor
[LGTM Timeline notifier]Timeline:
|
zimulala
approved these changes
Sep 30, 2024
Contributor
|
@zimulala: adding LGTM is restricted to approvers and reviewers in OWNERS files. DetailsIn response to this: Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: JaySon-Huang, zanmato1984, zimulala The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This was referenced Sep 30, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #9032
Problem Summary:
What is changed and how it works?
Check List
Tests
Side effects
Documentation
Release note