Skip to content

feat: Add support for ORC writer#15300

Closed
wypb wants to merge 1 commit intofacebookincubator:mainfrom
wypb:orc_writer_v2
Closed

feat: Add support for ORC writer#15300
wypb wants to merge 1 commit intofacebookincubator:mainfrom
wypb:orc_writer_v2

Conversation

@wypb
Copy link
Copy Markdown
Contributor

@wypb wypb commented Oct 28, 2025 •
edited
Loading

Part of #12487. Metadata changes have been merged: #12616.

This PR add support for ORC native writer:

  1. Support all Velox data types
  2. Support RleEncoderV2 encoding writing
  3. Support stripe-level statistics
  4. velox/dwio/orc/tests/E2EWriterTest.cpp file adds verification of reading and writing orc for all data types
  5. velox/dwio/orc/tests/E2EFilterTest.cpp file adds end-to-end read and write tests with filters
TYPE WRITE ENCODING STRIPE STATISTICS FILE STATISTICS INDEX
RLE v1 RLE v2 DICT
BOOLEAN ✅ ❌ ❌ ✅ ✅ ✅
FLOAT ✅ ❌ ❌ ✅ ✅ ✅
DOUBLE ✅ ❌ ❌ ✅ ✅ ✅
BYTE ✅ ❌ ❌ ✅ ✅ ✅
DATE ✅ ✅ ❌ ✅ ✅ ✅
SHORT ✅ ✅ ❌ ✅ ✅ ✅
INT ✅ ✅ ❌ ✅ ✅ ✅
LONG ✅ ✅ ❌ ✅ ✅ ✅
SHORT DECIMAL ✅ ✅ ❌ ✅ ✅ ✅
LONG DECIMAL ✅ ✅ ❌ ✅ ✅ ✅
BINARY ✅ ✅ ❌ ✅ ✅ ✅
TIMESTAMP ✅ ✅ ❌ ✅ ✅ ✅
VARCHAR ✅ ✅ ✅ ✅ ✅ ✅
STRING ✅ ✅ ✅ ✅ ✅ ✅
LIST ✅ ✅ ❌ ✅ ✅ ✅
MAP ✅ ✅ ❌ ✅ ✅ ✅
STRUCT ✅ ❌ ❌ ✅ ✅ ✅

CC: @Yuhta @majetideepak @rui-mo @HuamengJiang

@wypb wypb requested a review from majetideepak as a code owner October 28, 2025 07:57
@netlify
Copy link
Copy Markdown

netlify Bot commented Oct 28, 2025 •
edited
Loading

✅ Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit b2d8c80
🔍 Latest deploy log https://app.netlify.com/projects/meta-velox/deploys/6900a272aac628000862c0da

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 28, 2025
@wypb wypb force-pushed the orc_writer_v2 branch 4 times, most recently from 49bae81 to e217b21 Compare October 28, 2025 08:53
@PingLiuPing PingLiuPing mentioned this pull request Oct 29, 2025
Closed
@Yuhta Yuhta requested review from HuamengJiang and zzhao0 October 30, 2025 18:19
@stale
Copy link
Copy Markdown

stale Bot commented Jan 28, 2026

This pull request has been automatically marked as stale because it has not had recent activity. If you'd still like this PR merged, please comment on the PR, make sure you've addressed reviewer comments, and rebase on the latest main. Thank you for your contributions!

@stale stale Bot added the stale label Jan 28, 2026
@stale stale Bot closed this Feb 12, 2026
@rui-mo rui-mo mentioned this pull request Feb 12, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@majetideepak majetideepak Awaiting requested review from majetideepak majetideepak is a code owner

@zzhao0 zzhao0 Awaiting requested review from zzhao0

@HuamengJiang HuamengJiang Awaiting requested review from HuamengJiang

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. stale

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@wypb