-
Notifications
You must be signed in to change notification settings - Fork 4.1k
backupccl: add Apache Parquet format to export #67710
Copy link
Copy link
Closed
Labels
A-disaster-recoveryC-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)O-communityOriginated from the communityOriginated from the communityT-disaster-recoveryX-blathers-triagedblathers was able to find an ownerblathers was able to find an owner
Description
Describe the solution you'd like
Ability to export to Apache Parquet format.
EXPORT INTO PARQUET
'azure://acme-co/customer-export-data?AZURE_ACCOUNT_KEY=hash&AZURE_ACCOUNT_NAME=acme-co'
FROM TABLE bank.customers;
Describe alternatives you've considered
Using CSV format
Additional context
Parquet stores the file schema in the file metadata. CSV files don't store file metadata, so readers need to either be supplied with the schema or the schema needs to be inferred. Supplying a schema is tedious and inferring a schema is error prone / expensive. A couple of large customers are asking for this feature.
Epic CRDB-10340
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
A-disaster-recoveryC-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)O-communityOriginated from the communityOriginated from the communityT-disaster-recoveryX-blathers-triagedblathers was able to find an ownerblathers was able to find an owner