-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Support npy as input format #49293
Copy link
Copy link
Closed
Labels
comp-formatsInput/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.).Input/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.).feature
Description
npy is often used to encode numpy data including embeddings as a arrays. Large embedding datasets are often distributed in this format. Propose we support as an input format.
A mapping to clickhouse rows needs to be discussed, but possibly first level is mapped to row and nested arrays become Array(Array.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
comp-formatsInput/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.).Input/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.).feature