[VL] parquet file metadata columns support in velox#3870
[VL] parquet file metadata columns support in velox#3870zhli1142015 merged 13 commits intoapache:mainfrom
Conversation
|
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/oap-project/gluten/issues Then could you also rename commit message and pull request title in the following format? See also: |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
7 similar comments
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
d19534a to
43fc8bc
Compare
|
Run Gluten Clickhouse CI |
1 similar comment
|
Run Gluten Clickhouse CI |
5753e50 to
79ba0da
Compare
|
Run Gluten Clickhouse CI |
2 similar comments
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
273bee9 to
913d780
Compare
|
Run Gluten Clickhouse CI |
3 similar comments
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
@zhouyuan and @FelixYBW could you help check why centos7-test fail and give some input ? Meanwhile could you help have a draft review to see if current implement for metadata column native support good to you , if the implement ok for you guys , i will sync with meta velox guys for velox part PR review. |
|
@gaoyangxiaozhu Thanks for providing this support. The implementation seems okay. Please do a rebase and go ahead with corresponding support in Velox first. |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks. |
…xiaozhu/gluten into gayangya/metadatacolumns
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
…xiaozhu/gluten into gayangya/metadatacolumns
|
Run Gluten Clickhouse CI |
|
various build issue , can you help re-trigger @yma11 / @zhouyuan / @zhli1142015 |
|
Run Gluten Clickhouse CI |
|
can we re-trigger again for failed job which all due to below @zhouyuan / @yma11 / @zhli1142015 |
…gayangya/metadatacolumns
|
Run Gluten Clickhouse CI |
|
can we merge the PR ? @zhli1142015 / @yma11 / @zhouyuan |
|
@gaoyangxiaozhu This is in good state to me. Will try with internal delta lake jenkins job |
|
@gaoyangxiaozhu seems there is some conflicts and you need a rebase. Thanks. |
…gayangya/metadatacolumns
|
Run Gluten Clickhouse CI |
done |
|
===== Performance report for TPCH SF2000 with Velox backend, for reference only ====
|
[VL] parquet file metadata columns support in velox. Co-authored-by: Zhen Li <zhli@microsoft.com>
[VL] parquet file metadata columns support in velox. Co-authored-by: Zhen Li <zhli@microsoft.com>
[VL] parquet file metadata columns support in velox. Co-authored-by: Zhen Li <zhli@microsoft.com>


What changes were proposed in this pull request?
Support file metadata column velox native access, it is requirement of Delta native support for Delta Microsoft Team asked.
Fixes: #2618
How was this patch tested?
Via UT and manually tests
dependency PR in velox is facebookincubator/velox#8800 and is merged
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)