Skip to content

[Flash-600] PageStorage GC only remove obsoleted data#297

Merged
JaySon-Huang merged 5 commits intopingcap:DeltaMergeEnginefrom
JaySon-Huang:FLASH-600
Oct 30, 2019
Merged

[Flash-600] PageStorage GC only remove obsoleted data#297
JaySon-Huang merged 5 commits intopingcap:DeltaMergeEnginefrom
JaySon-Huang:FLASH-600

Conversation

@JaySon-Huang
Copy link
Contributor

@JaySon-Huang JaySon-Huang commented Oct 29, 2019

  • PageStorage
    • gcRemoveObsoleteFiles -> gcRemoveObsoleteData. Only remove PageFile's data.
    • Another gc for PageFile's meta will be done later(FLASH-530)
  • PageFile now have 3 state
    • Formal: written by PageStorage or generate from GC thread
    • Temp: writing by PageStorage GC thread
    • Legacy: PageStorage GC thread have already removed the data, just left meta for recover
  • WriteBatch && PageEntriesEdit
    • Add a new write type(MOVE_NORMAL_PAGE) to indicate that we only move the normal page to new PageEntry rather than putting new page

    Note that PUT means that we put a new page and need a ref-page with same page id. And MOVE_NORMAL_PAGE means that we move a exist page to a new PageFile.

@JaySon-Huang
Copy link
Contributor Author

/run-integration-tests

@flowbehappy
Copy link
Contributor

LGTM

@JaySon-Huang JaySon-Huang merged this pull request into pingcap:DeltaMergeEngine Oct 30, 2019
@JaySon-Huang JaySon-Huang deleted the FLASH-600 branch October 30, 2019 10:56
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Oct 30, 2019
* Add test case for migrate ref-page to ref-page

* check if ref-page is valid

* only remove PageFile's data

* Add MOVE_NORMAL_PAGE for only move normal page

* enhance utils_get_valid_pages
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Nov 1, 2019
* Add test case for migrate ref-page to ref-page

* check if ref-page is valid

* only remove PageFile's data

* Add MOVE_NORMAL_PAGE for only move normal page

* enhance utils_get_valid_pages
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Nov 1, 2019
* Add test case for migrate ref-page to ref-page

* check if ref-page is valid

* only remove PageFile's data

* Add MOVE_NORMAL_PAGE for only move normal page

* enhance utils_get_valid_pages
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Nov 5, 2019
* Add test case for migrate ref-page to ref-page

* check if ref-page is valid

* only remove PageFile's data

* Add MOVE_NORMAL_PAGE for only move normal page

* enhance utils_get_valid_pages
CalvinNeo pushed a commit to CalvinNeo/tiflash that referenced this pull request Sep 23, 2024
Signed-off-by: yongman <yming0221@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants