[Feature] doris cross-cluster query #57898
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
c045b8e to
4b7d8a7
Compare
Contributor
Author
|
run buildall |
Contributor
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
FE UT Coverage ReportIncrement line coverage |
TPC-H: Total hot run time: 34370 ms |
TPC-DS: Total hot run time: 186990 ms |
ClickBench: Total hot run time: 28.17 s |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
FE Regression Coverage ReportIncrement line coverage |
4b7d8a7 to
07d57c7
Compare
Contributor
Author
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
07d57c7 to
6ebf97d
Compare
Contributor
Author
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
FE UT Coverage ReportIncrement line coverage |
TPC-H: Total hot run time: 34344 ms |
Contributor
Author
|
run external |
Contributor
Author
|
run p0 |
TPC-DS: Total hot run time: 187199 ms |
ClickBench: Total hot run time: 28.32 s |
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
FE Regression Coverage ReportIncrement line coverage |
morningman
approved these changes
Nov 30, 2025
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
TPC-H: Total hot run time: 34035 ms |
TPC-DS: Total hot run time: 181591 ms |
ClickBench: Total hot run time: 28.5 s |
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
924060929
approved these changes
Dec 1, 2025
github-actions bot
pushed a commit
that referenced
this pull request
Dec 1, 2025
Related issue: #57884 ``` MySQL [demo]> show frontends; +-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+ | Name | Host | EditLogPort | HttpPort | QueryPort | RpcPort | ArrowFlightSqlPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastStartTime | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected | LiveSince | +-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+ | fe_a7c0b6d8_82c2_48f0_8220_fb65dd18be69 | 10.37.75.124 | 9010 | 8030 | 9030 | 9020 | 8070 | FOLLOWER | true | 742250121 | true | true | 2409 | 2025-11-11 14:42:16 | 2025-11-11 14:44:06 | true | | doris-0.0.0-009c3b552a | Yes | 2025-11-11 14:42:16 | +-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+ 1 row in set (0.016 sec) MySQL [demo]> show catalog edoris; +-----------------+-------------------------------+ | Key | Value | +-----------------+-------------------------------+ | create_time | 2025-11-11 11:25:33.488106853 | | fe_arrow_hosts | 10.37.103.28:8070 | | fe_http_hosts | 10.37.103.28:8030 | | fe_thrift_hosts | 10.37.103.28:9020 | | password | *XXX | | type | doris | | use_meta_cache | true | | user | test | +-----------------+-------------------------------+ 8 rows in set (0.002 sec) MySQL [demo]> select * from inner_table; +----------+--------+ | log_type | reason | +----------+--------+ | 2 | inner2 | | 3 | inner3 | | 4 | inner4 | +----------+--------+ 3 rows in set (0.032 sec) MySQL [demo]> select * from edoris.external.example_tbl_duplicate; +---------------------+----------+------------+-----------+-------+---------------------+ | log_time | log_type | error_code | error_msg | op_id | op_time | +---------------------+----------+------------+-----------+-------+---------------------+ | 2024-11-01 00:00:00 | 2 | 2 | timeout | 12 | 2024-11-01 01:00:00 | +---------------------+----------+------------+-----------+-------+---------------------+ 1 row in set (0.059 sec) MySQL [demo]> select * from inner_table a join edoris.external.example_tbl_duplicate b on (a.log_type = b.log_type); +----------+--------+---------------------+----------+------------+-----------+-------+---------------------+ | log_type | reason | log_time | log_type | error_code | error_msg | op_id | op_time | +----------+--------+---------------------+----------+------------+-----------+-------+---------------------+ | 2 | inner2 | 2024-11-01 00:00:00 | 2 | 2 | timeout | 12 | 2024-11-01 01:00:00 | +----------+--------+---------------------+----------+------------+-----------+-------+---------------------+ 1 row in set (0.050 sec) MySQL [demo]> explain select * from inner_table a join edoris.external.example_tbl_duplicate b on (a.log_type = b.log_type) where error_code=2; +-------------------------------------------------------------------------------------------------------------------------------------------+ | Explain String(Nereids Planner) | +-------------------------------------------------------------------------------------------------------------------------------------------+ | PLAN FRAGMENT 0 | | OUTPUT EXPRS: | | log_type[#16] | | reason[#17] | | log_time[#18] | | log_type[#19] | | error_code[#20] | | error_msg[#21] | | op_id[#22] | | op_time[#23] | | PARTITION: HASH_PARTITIONED: log_type[#6] | | | | HAS_COLO_PLAN_NODE: false | | | | VRESULT SINK | | MYSQL_PROTOCOL | | | | 3:VHASH JOIN(200) | | | join op: INNER JOIN(BROADCAST)[] | | | equal join conjunct: (log_type[#6] = log_type[#1]) | | | cardinality=3 | | | vec output tuple id: 3 | | | output tuple id: 3 | | | vIntermediate tuple ids: 2 | | | hash output slot ids: 0 1 2 3 4 5 6 7 | | | runtime filters: RF000[min_max] <- log_type[#1](1/1/1048576), RF001[in_or_bloom] <- log_type[#1](1/1/1048576) | | | final projections: log_type[#8], reason[#9], log_time[#10], log_type[#11], error_code[#12], error_msg[#13], op_id[#14], op_time[#15] | | | final project output tuple id: 3 | | | distribute expr lists: log_type[#6] | | | distribute expr lists: | | | | | |----1:VEXCHANGE | | | offset: 0 | | | distribute expr lists: log_type[#1] | | | | | 2:VOlapScanNode(187) | | TABLE: demo.inner_table(inner_table), PREAGGREGATION: ON | | partitions=1/1 (inner_table) | | tablets=1/1, tabletList=1762832514491 | | cardinality=3, avgRowSize=901.6666, numNodes=1 | | pushAggOp=NONE | | runtime filters: RF000[min_max] -> log_type[#6], RF001[in_or_bloom] -> log_type[#6] | | | | PLAN FRAGMENT 1 | | | | PARTITION: HASH_PARTITIONED: log_type[#1] | | | | HAS_COLO_PLAN_NODE: false | | | | STREAM DATA SINK | | EXCHANGE ID: 01 | | UNPARTITIONED | | | | 0:VOlapScanNode(188) | | TABLE: external.example_tbl_duplicate(example_tbl_duplicate), PREAGGREGATION: ON | | PREDICATES: (error_code[#2] = 2) | | partitions=1/1 (example_tbl_duplicate) | | tablets=1/1, tabletList=1762481736238 | | cardinality=1, avgRowSize=7425.0, numNodes=1 | | pushAggOp=NONE | | | | | | | | ========== STATISTICS ========== | | planed with unknown column statistics | +-------------------------------------------------------------------------------------------------------------------------------------------+ 65 rows in set (0.040 sec) ```
morningman
pushed a commit
that referenced
this pull request
Dec 2, 2025
Related issue: #57884 ``` MySQL [demo]> show frontends; +-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+ | Name | Host | EditLogPort | HttpPort | QueryPort | RpcPort | ArrowFlightSqlPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastStartTime | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected | LiveSince | +-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+ | fe_a7c0b6d8_82c2_48f0_8220_fb65dd18be69 | 10.37.75.124 | 9010 | 8030 | 9030 | 9020 | 8070 | FOLLOWER | true | 742250121 | true | true | 2409 | 2025-11-11 14:42:16 | 2025-11-11 14:44:06 | true | | doris-0.0.0-009c3b552a | Yes | 2025-11-11 14:42:16 | +-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+ 1 row in set (0.016 sec) MySQL [demo]> show catalog edoris; +-----------------+-------------------------------+ | Key | Value | +-----------------+-------------------------------+ | create_time | 2025-11-11 11:25:33.488106853 | | fe_arrow_hosts | 10.37.103.28:8070 | | fe_http_hosts | 10.37.103.28:8030 | | fe_thrift_hosts | 10.37.103.28:9020 | | password | *XXX | | type | doris | | use_meta_cache | true | | user | test | +-----------------+-------------------------------+ 8 rows in set (0.002 sec) MySQL [demo]> select * from inner_table; +----------+--------+ | log_type | reason | +----------+--------+ | 2 | inner2 | | 3 | inner3 | | 4 | inner4 | +----------+--------+ 3 rows in set (0.032 sec) MySQL [demo]> select * from edoris.external.example_tbl_duplicate; +---------------------+----------+------------+-----------+-------+---------------------+ | log_time | log_type | error_code | error_msg | op_id | op_time | +---------------------+----------+------------+-----------+-------+---------------------+ | 2024-11-01 00:00:00 | 2 | 2 | timeout | 12 | 2024-11-01 01:00:00 | +---------------------+----------+------------+-----------+-------+---------------------+ 1 row in set (0.059 sec) MySQL [demo]> select * from inner_table a join edoris.external.example_tbl_duplicate b on (a.log_type = b.log_type); +----------+--------+---------------------+----------+------------+-----------+-------+---------------------+ | log_type | reason | log_time | log_type | error_code | error_msg | op_id | op_time | +----------+--------+---------------------+----------+------------+-----------+-------+---------------------+ | 2 | inner2 | 2024-11-01 00:00:00 | 2 | 2 | timeout | 12 | 2024-11-01 01:00:00 | +----------+--------+---------------------+----------+------------+-----------+-------+---------------------+ 1 row in set (0.050 sec) MySQL [demo]> explain select * from inner_table a join edoris.external.example_tbl_duplicate b on (a.log_type = b.log_type) where error_code=2; +-------------------------------------------------------------------------------------------------------------------------------------------+ | Explain String(Nereids Planner) | +-------------------------------------------------------------------------------------------------------------------------------------------+ | PLAN FRAGMENT 0 | | OUTPUT EXPRS: | | log_type[#16] | | reason[#17] | | log_time[#18] | | log_type[#19] | | error_code[#20] | | error_msg[#21] | | op_id[#22] | | op_time[#23] | | PARTITION: HASH_PARTITIONED: log_type[#6] | | | | HAS_COLO_PLAN_NODE: false | | | | VRESULT SINK | | MYSQL_PROTOCOL | | | | 3:VHASH JOIN(200) | | | join op: INNER JOIN(BROADCAST)[] | | | equal join conjunct: (log_type[#6] = log_type[#1]) | | | cardinality=3 | | | vec output tuple id: 3 | | | output tuple id: 3 | | | vIntermediate tuple ids: 2 | | | hash output slot ids: 0 1 2 3 4 5 6 7 | | | runtime filters: RF000[min_max] <- log_type[#1](1/1/1048576), RF001[in_or_bloom] <- log_type[#1](1/1/1048576) | | | final projections: log_type[#8], reason[#9], log_time[#10], log_type[#11], error_code[#12], error_msg[#13], op_id[#14], op_time[#15] | | | final project output tuple id: 3 | | | distribute expr lists: log_type[#6] | | | distribute expr lists: | | | | | |----1:VEXCHANGE | | | offset: 0 | | | distribute expr lists: log_type[#1] | | | | | 2:VOlapScanNode(187) | | TABLE: demo.inner_table(inner_table), PREAGGREGATION: ON | | partitions=1/1 (inner_table) | | tablets=1/1, tabletList=1762832514491 | | cardinality=3, avgRowSize=901.6666, numNodes=1 | | pushAggOp=NONE | | runtime filters: RF000[min_max] -> log_type[#6], RF001[in_or_bloom] -> log_type[#6] | | | | PLAN FRAGMENT 1 | | | | PARTITION: HASH_PARTITIONED: log_type[#1] | | | | HAS_COLO_PLAN_NODE: false | | | | STREAM DATA SINK | | EXCHANGE ID: 01 | | UNPARTITIONED | | | | 0:VOlapScanNode(188) | | TABLE: external.example_tbl_duplicate(example_tbl_duplicate), PREAGGREGATION: ON | | PREDICATES: (error_code[#2] = 2) | | partitions=1/1 (example_tbl_duplicate) | | tablets=1/1, tabletList=1762481736238 | | cardinality=1, avgRowSize=7425.0, numNodes=1 | | pushAggOp=NONE | | | | | | | | ========== STATISTICS ========== | | planed with unknown column statistics | +-------------------------------------------------------------------------------------------------------------------------------------------+ 65 rows in set (0.040 sec) ```
5 tasks
morningman
added a commit
to apache/doris-website
that referenced
this pull request
Dec 5, 2025
…y method (#3138) apache/doris#57898 ## Versions - [ x ] dev - [ x ] 4.x - [ ] 3.x - [ ] 2.1 ## Languages - [ x ] Chinese - [ ] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built --------- Co-authored-by: zhangshixin.1024 <zhangshixin.1024@bytedance.com> Co-authored-by: Mingyu Chen (Rayner) <yunyou@selectdb.com>
yiguolei
pushed a commit
that referenced
this pull request
Dec 9, 2025
nagisa-kunhah
pushed a commit
to nagisa-kunhah/doris
that referenced
this pull request
Dec 14, 2025
Related issue: apache#57884 ``` MySQL [demo]> show frontends; +-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+ | Name | Host | EditLogPort | HttpPort | QueryPort | RpcPort | ArrowFlightSqlPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastStartTime | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected | LiveSince | +-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+ | fe_a7c0b6d8_82c2_48f0_8220_fb65dd18be69 | 10.37.75.124 | 9010 | 8030 | 9030 | 9020 | 8070 | FOLLOWER | true | 742250121 | true | true | 2409 | 2025-11-11 14:42:16 | 2025-11-11 14:44:06 | true | | doris-0.0.0-009c3b552a | Yes | 2025-11-11 14:42:16 | +-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+ 1 row in set (0.016 sec) MySQL [demo]> show catalog edoris; +-----------------+-------------------------------+ | Key | Value | +-----------------+-------------------------------+ | create_time | 2025-11-11 11:25:33.488106853 | | fe_arrow_hosts | 10.37.103.28:8070 | | fe_http_hosts | 10.37.103.28:8030 | | fe_thrift_hosts | 10.37.103.28:9020 | | password | *XXX | | type | doris | | use_meta_cache | true | | user | test | +-----------------+-------------------------------+ 8 rows in set (0.002 sec) MySQL [demo]> select * from inner_table; +----------+--------+ | log_type | reason | +----------+--------+ | 2 | inner2 | | 3 | inner3 | | 4 | inner4 | +----------+--------+ 3 rows in set (0.032 sec) MySQL [demo]> select * from edoris.external.example_tbl_duplicate; +---------------------+----------+------------+-----------+-------+---------------------+ | log_time | log_type | error_code | error_msg | op_id | op_time | +---------------------+----------+------------+-----------+-------+---------------------+ | 2024-11-01 00:00:00 | 2 | 2 | timeout | 12 | 2024-11-01 01:00:00 | +---------------------+----------+------------+-----------+-------+---------------------+ 1 row in set (0.059 sec) MySQL [demo]> select * from inner_table a join edoris.external.example_tbl_duplicate b on (a.log_type = b.log_type); +----------+--------+---------------------+----------+------------+-----------+-------+---------------------+ | log_type | reason | log_time | log_type | error_code | error_msg | op_id | op_time | +----------+--------+---------------------+----------+------------+-----------+-------+---------------------+ | 2 | inner2 | 2024-11-01 00:00:00 | 2 | 2 | timeout | 12 | 2024-11-01 01:00:00 | +----------+--------+---------------------+----------+------------+-----------+-------+---------------------+ 1 row in set (0.050 sec) MySQL [demo]> explain select * from inner_table a join edoris.external.example_tbl_duplicate b on (a.log_type = b.log_type) where error_code=2; +-------------------------------------------------------------------------------------------------------------------------------------------+ | Explain String(Nereids Planner) | +-------------------------------------------------------------------------------------------------------------------------------------------+ | PLAN FRAGMENT 0 | | OUTPUT EXPRS: | | log_type[apache#16] | | reason[apache#17] | | log_time[apache#18] | | log_type[apache#19] | | error_code[apache#20] | | error_msg[apache#21] | | op_id[apache#22] | | op_time[apache#23] | | PARTITION: HASH_PARTITIONED: log_type[apache#6] | | | | HAS_COLO_PLAN_NODE: false | | | | VRESULT SINK | | MYSQL_PROTOCOL | | | | 3:VHASH JOIN(200) | | | join op: INNER JOIN(BROADCAST)[] | | | equal join conjunct: (log_type[apache#6] = log_type[apache#1]) | | | cardinality=3 | | | vec output tuple id: 3 | | | output tuple id: 3 | | | vIntermediate tuple ids: 2 | | | hash output slot ids: 0 1 2 3 4 5 6 7 | | | runtime filters: RF000[min_max] <- log_type[apache#1](1/1/1048576), RF001[in_or_bloom] <- log_type[apache#1](1/1/1048576) | | | final projections: log_type[apache#8], reason[apache#9], log_time[apache#10], log_type[apache#11], error_code[apache#12], error_msg[apache#13], op_id[apache#14], op_time[apache#15] | | | final project output tuple id: 3 | | | distribute expr lists: log_type[apache#6] | | | distribute expr lists: | | | | | |----1:VEXCHANGE | | | offset: 0 | | | distribute expr lists: log_type[apache#1] | | | | | 2:VOlapScanNode(187) | | TABLE: demo.inner_table(inner_table), PREAGGREGATION: ON | | partitions=1/1 (inner_table) | | tablets=1/1, tabletList=1762832514491 | | cardinality=3, avgRowSize=901.6666, numNodes=1 | | pushAggOp=NONE | | runtime filters: RF000[min_max] -> log_type[apache#6], RF001[in_or_bloom] -> log_type[apache#6] | | | | PLAN FRAGMENT 1 | | | | PARTITION: HASH_PARTITIONED: log_type[apache#1] | | | | HAS_COLO_PLAN_NODE: false | | | | STREAM DATA SINK | | EXCHANGE ID: 01 | | UNPARTITIONED | | | | 0:VOlapScanNode(188) | | TABLE: external.example_tbl_duplicate(example_tbl_duplicate), PREAGGREGATION: ON | | PREDICATES: (error_code[apache#2] = 2) | | partitions=1/1 (example_tbl_duplicate) | | tablets=1/1, tabletList=1762481736238 | | cardinality=1, avgRowSize=7425.0, numNodes=1 | | pushAggOp=NONE | | | | | | | | ========== STATISTICS ========== | | planed with unknown column statistics | +-------------------------------------------------------------------------------------------------------------------------------------------+ 65 rows in set (0.040 sec) ```
16 tasks
morningman
added a commit
that referenced
this pull request
Dec 22, 2025
### What problem does this PR solve? Related PR: #57898 Problem Summary: The `getBackendMeta` does not need to call to Master FE. If use specified non-master FE address in Doris Catalog, it will result in dead loop
github-actions bot
pushed a commit
that referenced
this pull request
Dec 22, 2025
### What problem does this PR solve? Related PR: #57898 Problem Summary: The `getBackendMeta` does not need to call to Master FE. If use specified non-master FE address in Doris Catalog, it will result in dead loop
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related issue: #57884