Change cumulative compaction for decoupling storage from compution #1576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

imay merged 11 commits into apache:master from chaoyli:master

Aug 13, 2019

Contributor

chaoyli commented Aug 2, 2019

Calculate cumulative point when loading tablet first time.
Simplify pick rowsets logic upon delete predicate.
Saving meta and modify rowsets only once after cumulative compaction.

imay requested changes

View reviewed changes

be/src/olap/base_compaction.h Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

be/src/olap/base_compaction.cpp Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

be/src/olap/base_compaction.cpp Outdated Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

be/src/olap/base_compaction.cpp Outdated Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

be/src/olap/base_compaction.cpp Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

be/src/olap/base_compaction.cpp Outdated Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

be/src/olap/cumulative_compaction.cpp Outdated Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

be/src/olap/cumulative_compaction.cpp Outdated Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

be/src/olap/tablet.cpp Outdated Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

be/src/olap/tablet.cpp Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

be/src/olap/tablet.cpp Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

be/src/olap/base_compaction.cpp Outdated Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

Contributor

yiguolei left a comment

some comments

yiguolei reviewed

View reviewed changes

be/src/olap/tablet.cpp Show resolved Hide resolved

yiguolei previously approved these changes

View reviewed changes

Contributor

yiguolei left a comment

LGTM

imay reviewed

View reviewed changes

be/src/olap/tablet_manager.cpp Show resolved Hide resolved

be/src/olap/base_compaction.cpp Outdated Show resolved Hide resolved

be/src/olap/tablet.cpp Show resolved Hide resolved

be/src/olap/storage_engine.cpp Show resolved Hide resolved

chaoyli dismissed yiguolei’s stale review via

cee8dc1

August 7, 2019 13:43

yiguolei reviewed

View reviewed changes

be/src/olap/tablet_manager.cpp Outdated Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

Contributor

yiguolei left a comment

some comments

yiguolei previously approved these changes

View reviewed changes

Contributor

yiguolei left a comment

LGTM

imay reviewed

View reviewed changes

be/src/olap/base_compaction.cpp Outdated Show resolved Hide resolved

be/src/olap/base_compaction.cpp Outdated Show resolved Hide resolved

be/src/olap/base_compaction.cpp Outdated Show resolved Hide resolved

be/src/olap/base_compaction.cpp Outdated Show resolved Hide resolved

chaoyli dismissed yiguolei’s stale review via

95e169f

August 8, 2019 13:44

imay reviewed

View reviewed changes

be/src/olap/base_compaction.cpp Outdated Show resolved Hide resolved

be/src/olap/compaction.h Show resolved Hide resolved

be/src/olap/base_compaction.cpp Outdated Show resolved Hide resolved

yiguolei reviewed

View reviewed changes

Contributor

yiguolei left a comment

LGTM

imay reviewed

View reviewed changes

be/src/olap/base_compaction.cpp Outdated Show resolved Hide resolved

be/src/olap/base_compaction.h Outdated Show resolved Hide resolved

xy720 and others added 5 commits

August 13, 2019 15:04


          Merge Help document to documentation (#1586)

86b6e74

Help document collation (integration of help and documentation documents)


          Change cumulative compaction for decoupling storage from compution

dbd5647

1. Calculate cumulative point when loading tablet first time.
2. Simplify pick rowsets logic upon delete predicate.
3. Saving meta and modify rowsets only once after cumulative compaction.


          Change base compaction for decoupling storage from compution

f75f7ef

1. Simplify pick rowsets logic.
2. Saving meta and modify rowsets only once after base compaction.


          Extract Compaction Class from CumulativeCompaction and BaseComapction.

3aca6bb

1. Add annotation to Compaction Class
2. Fix set cumulative_layer_point bug before cumulative compaction completion.


          Fix calculate cumulative point bug

059fd2b

chaoyli added 6 commits

August 13, 2019 15:04


          Add mutex to thread-safe

3dcc2f8


          Move calculate_cumulative_point() logic to cumulative compaction

c96e1b4


          Change log level

7a5c865


          Extract common code to base class

afe86a7


          Make function only used in compaction protected

8e63ce6


          Move codes in cppl file to header file

43ce5fb

imay approved these changes

View reviewed changes

Contributor

imay left a comment

LGTM

imay merged commit dcb7572 into apache:master

luwei16 added a commit to luwei16/incubator-doris that referenced this pull request


          [feature](selectdb-cloud) Report an error directly when the cluster i…

357defc

…s empty (apache#1576)

* Report an error directly when the cluster is empty
* fix regression case

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet