Skip to content

Binary compatibility version for DeltaMerge's chunk; Fix disappear of PageStorage del meta#257

Merged
JaySon-Huang merged 6 commits intopingcap:DeltaMergeEnginefrom
JaySon-Huang:dm-pagestorage-fix
Sep 27, 2019
Merged

Binary compatibility version for DeltaMerge's chunk; Fix disappear of PageStorage del meta#257
JaySon-Huang merged 6 commits intopingcap:DeltaMergeEnginefrom
JaySon-Huang:dm-pagestorage-fix

Conversation

@JaySon-Huang
Copy link
Contributor

@JaySon-Huang JaySon-Huang commented Sep 27, 2019

DeltaMerge

  • Add binary compatibility version for chunk. We can update serialized chunk format in the future without breaking old data.

PageStorage

  • Migrate DelPage meta in doing GC.
  • Add name to PageStorage, so that we can tell which storage is happen to error when there is multiple DeltaMerge tables.
  • Remove getMaxId of PageStorage. We just need the max Page ID when we load PageStorage from disk, removing this API so that we can ignore max ID when putting new Page.

@JaySon-Huang
Copy link
Contributor Author

/run-integration-tests

@flowbehappy
Copy link
Contributor

LGTM

@JaySon-Huang
Copy link
Contributor Author

/run-integration-tests

@JaySon-Huang
Copy link
Contributor Author

/run-integration-tests

@JaySon-Huang JaySon-Huang merged commit 779acfa into pingcap:DeltaMergeEngine Sep 27, 2019
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Oct 17, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Oct 18, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit that referenced this pull request Oct 22, 2019
… PageStorage del meta (#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Oct 23, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Oct 30, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Nov 1, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Nov 1, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Nov 5, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang pushed a commit that referenced this pull request Aug 6, 2024
Signed-off-by: Wish <breezewish@outlook.com>
JaySon-Huang pushed a commit that referenced this pull request Aug 6, 2024
Signed-off-by: Wish <breezewish@outlook.com>
JaySon-Huang pushed a commit to JaySon-Huang/tiflash that referenced this pull request Aug 7, 2024
Signed-off-by: Wish <breezewish@outlook.com>
Lloyd-Pottiger pushed a commit to Lloyd-Pottiger/tiflash that referenced this pull request Aug 30, 2024
Signed-off-by: Wish <breezewish@outlook.com>
Lloyd-Pottiger pushed a commit to Lloyd-Pottiger/tiflash that referenced this pull request Aug 30, 2024
Signed-off-by: Wish <breezewish@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants