Add `space_map` into PageStorage V3 by jiaqizho · Pull Request #3493 · pingcap/tiflash

jiaqizho · 2021-11-22T10:30:14Z

What problem does this PR solve?

Issue Number:

Problem Summary:

What is changed and how it works?

Proposal:

What's Changed:

split from [POC]New page storage #3197
now, space_map support free location
- it can split range when calling the mark method with a no nearby block.
- it can merge range when unmark the range with a nearby block.
Added some cmake as PageStorage V3 beginning.
Added some gtests for space_map.

Similar scope mergers have not yet been done. For this, there will create a new PR in the future.

Related changes

PR to update pingcap/docs/pingcap/docs-cn:
Need to cherry-pick to the release branch:

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Release note

None

ti-chi-bot · 2021-11-22T10:30:15Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

JaySon-Huang

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Details

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

dbms/src/Storages/Page/CMakeLists.txt

dbms/src/Storages/Page/V3/spacemap/bits.h

Signed-off-by: JaySon-Huang <tshent@qq.com>

Refactor SpaceMap interface

dbms/src/Storages/Page/V3/spacemap/SpaceMapRBTree.cpp

dbms/src/Storages/Page/V3/spacemap/SpaceMapSTDMap.h

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h

dbms/src/Storages/Page/V3/spacemap/SpaceMapSTDMap.h

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h

jiaqizho · 2021-12-06T03:54:56Z

/run-all-tests

sre-bot · 2021-12-06T04:08:30Z

Coverage detail: https://ci-internal.pingcap.net/job/tics_ghpr_unit_test/258/cobertura/
(Coverage detail url is limited office network access)

lines: 37.6% (44765 out of 119179)
branches: 5.5% (67446 out of 1217007)

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h

Signed-off-by: JaySon-Huang <tshent@qq.com>

sre-bot · 2021-12-08T12:18:33Z

Coverage detail: https://ci-internal.pingcap.net/job/tics_ghpr_unit_test/292/cobertura/
(Coverage detail url is limited office network access)

lines: 41.0% (45541 out of 111197)
branches: 5.7% (68562 out of 1201804)

Signed-off-by: JaySon-Huang <tshent@qq.com>

JaySon-Huang

Check out the changes in https://github.com/jiaqizho/tics/pull/5 and resolve the below comments.

The line coverage for files under spacemap is only 67%. I hope it can be greater than 80% before merging. https://ci-internal.pingcap.net/job/tics_ghpr_unit_test/292/cobertura/dbms_src_Storages_Page_V3_spacemap/

dbms/src/Storages/Page/V3/spacemap/SpaceMapSTDMap.h

dbms/src/Storages/Page/V3/spacemap/SpaceMapRBTree.cpp

SpaceMap interface cleanup

Co-authored-by: JaySon <jayson.hjs@gmail.com>

jiaqizho · 2021-12-09T02:23:37Z

/run-unit-test

sre-bot · 2021-12-09T02:37:36Z

Coverage detail: https://ci-internal.pingcap.net/job/tics_ghpr_unit_test/298/cobertura/
(Coverage detail url is limited office network access)

lines: 41.1% (45729 out of 111243)
branches: 5.7% (68776 out of 1202106)

JaySon-Huang

Fix the comments

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h

JaySon-Huang

LGTM

Co-authored-by: JaySon <jayson.hjs@gmail.com>

jiaqizho · 2021-12-09T03:23:07Z

/merge

ti-chi-bot · 2021-12-09T03:23:07Z

@jiaqizho: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2021-12-09T03:23:13Z

This pull request has been accepted and is ready to merge.

Details

Commit hash: 00bf672

sre-bot · 2021-12-09T03:36:00Z

Coverage detail: https://ci-internal.pingcap.net/job/tics_ghpr_unit_test/299/cobertura/
(Coverage detail url is limited office network access)

lines: 41.0% (45652 out of 111243)
branches: 5.7% (68706 out of 1202106)

ti-chi-bot added the release-note-none Denotes a PR that doesn't merit a release note. label Nov 22, 2021

jiaqizho requested a review from JaySon-Huang November 22, 2021 10:30

ti-chi-bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 22, 2021

JaySon-Huang reviewed Nov 23, 2021

View reviewed changes

dbms/src/Storages/Page/CMakeLists.txt Outdated Show resolved Hide resolved

ti-chi-bot requested review from ilovesoup and solotzg and removed request for ilovesoup and solotzg November 23, 2021 05:07

add space map into pagestorage v3

c8f5b09

JaySon-Huang reviewed Nov 26, 2021

View reviewed changes

dbms/src/Storages/Page/V3/spacemap/bits.h Outdated Show resolved Hide resolved

jiaqizho added 2 commits November 29, 2021 10:26

Merge branch 'master' into bak-space-map

a60594b

[WIP]just for read

48ac64c

JaySon-Huang changed the title ~~Add space_map into PageStorage V3~~ [WIP] Add space_map into PageStorage V3 Nov 29, 2021

ti-chi-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 29, 2021

JaySon-Huang and others added 4 commits November 29, 2021 18:22

Try refactor interface

9d3031d

Signed-off-by: JaySon-Huang <tshent@qq.com>

Add spacemap.cpp

98ec4ce

Signed-off-by: JaySon-Huang <tshent@qq.com>

Merge pull request #3 from JaySon-Huang/add-space-map

2e03320

Refactor SpaceMap interface

finished std map implement

04776e1

JaySon-Huang reviewed Nov 30, 2021

View reviewed changes

dbms/src/Storages/Page/V3/spacemap/SpaceMapRBTree.cpp Outdated Show resolved Hide resolved

dbms/src/Storages/Page/V3/spacemap/SpaceMapSTDMap.h Outdated Show resolved Hide resolved

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h Outdated Show resolved Hide resolved

JaySon-Huang reviewed Nov 30, 2021

View reviewed changes

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h Outdated Show resolved Hide resolved

JaySon-Huang reviewed Dec 3, 2021

View reviewed changes

dbms/src/Storages/Page/V3/spacemap/SpaceMapSTDMap.h Outdated Show resolved Hide resolved

dbms/src/Storages/Page/V3/spacemap/SpaceMapSTDMap.h Outdated Show resolved Hide resolved

JinheLin reviewed Dec 3, 2021

View reviewed changes

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h Show resolved Hide resolved

JinheLin reviewed Dec 3, 2021

View reviewed changes

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h Show resolved Hide resolved

update

93ee120

jiaqizho changed the title ~~[WIP] Add space_map into PageStorage V3~~ Add space_map into PageStorage V3 Dec 6, 2021

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 6, 2021

JaySon-Huang reviewed Dec 6, 2021

View reviewed changes

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h Outdated Show resolved Hide resolved

JaySon-Huang added 2 commits December 8, 2021 20:13

Abstract out finding less or equal from std::map

a352497

Signed-off-by: JaySon-Huang <tshent@qq.com>

Cleanup redunt space in loggings

70c7c63

Signed-off-by: JaySon-Huang <tshent@qq.com>

JaySon-Huang added 4 commits December 8, 2021 20:23

Remove useless new/free virtual functions

f51117c

Signed-off-by: JaySon-Huang <tshent@qq.com>

Keep include header path style

7bbb503

Signed-off-by: JaySon-Huang <tshent@qq.com>

Cleanup interface

5911eac

Signed-off-by: JaySon-Huang <tshent@qq.com>

Format files

a3ed7df

Signed-off-by: JaySon-Huang <tshent@qq.com>

JaySon-Huang reviewed Dec 8, 2021

View reviewed changes

jiaqizho and others added 5 commits December 8, 2021 22:20

Merge pull request #5 from JaySon-Huang/add-space-map

6809c43

SpaceMap interface cleanup

Update dbms/src/Storages/Page/V3/spacemap/SpaceMapSTDMap.h

d04e793

Co-authored-by: JaySon <jayson.hjs@gmail.com>

update

a5e0ac8

fix some error

9b5684e

add tests

0ab02ca

pingcap deleted a comment from sre-bot Dec 9, 2021

JaySon-Huang reviewed Dec 9, 2021

View reviewed changes

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h Show resolved Hide resolved

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h Show resolved Hide resolved

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h Show resolved Hide resolved

dbms/src/Storages/Page/V3/spacemap/SpaceMap.h Show resolved Hide resolved

JaySon-Huang approved these changes Dec 9, 2021

View reviewed changes

Update dbms/src/Storages/Page/V3/spacemap/SpaceMap.h

a639b66

Co-authored-by: JaySon <jayson.hjs@gmail.com>

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Dec 9, 2021

jiaqizho and others added 3 commits December 9, 2021 11:22

Update dbms/src/Storages/Page/V3/spacemap/SpaceMap.h

39ce4d8

Co-authored-by: JaySon <jayson.hjs@gmail.com>

Update dbms/src/Storages/Page/V3/spacemap/SpaceMap.h

466f5b1

Co-authored-by: JaySon <jayson.hjs@gmail.com>

Update dbms/src/Storages/Page/V3/spacemap/SpaceMap.h

00bf672

Co-authored-by: JaySon <jayson.hjs@gmail.com>

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Dec 9, 2021

ti-chi-bot merged commit 4dc0526 into pingcap:master Dec 9, 2021

JaySon-Huang deleted the add-space-map branch December 9, 2021 04:38

Conversation

jiaqizho commented Nov 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What is changed and how it works?

Related changes

Check List

Release note

Uh oh!

ti-chi-bot commented Nov 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jiaqizho commented Dec 6, 2021

Uh oh!

sre-bot commented Dec 6, 2021

Uh oh!

Uh oh!

sre-bot commented Dec 8, 2021

Uh oh!

JaySon-Huang left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jiaqizho commented Dec 9, 2021

Uh oh!

sre-bot commented Dec 9, 2021

Uh oh!

JaySon-Huang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JaySon-Huang left a comment

Choose a reason for hiding this comment

Uh oh!

jiaqizho commented Dec 9, 2021

Uh oh!

ti-chi-bot commented Dec 9, 2021

Uh oh!

ti-chi-bot commented Dec 9, 2021

Uh oh!

sre-bot commented Dec 9, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jiaqizho commented Nov 22, 2021 •

edited

Loading

ti-chi-bot commented Nov 22, 2021 •

edited

Loading

JaySon-Huang left a comment •

edited

Loading