Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Something I discovered while working on code base at Coralogix and in Ballista.
arrow_flight::utils::flight_data_from_arrow_batch is used as method to convert a singleRecordBatch to FlightData.
This method extracts both the dictionaries as the RecordBatches.
While convenient, if the dictionary is the shared across batches, the caller will probably send the FlightData for these dictionaries over and over again over in the stream.
Describe the solution you'd like
Mark the method as deprecated to encourage users to use the IpcDataGenerator API instead.
https://docs.rs/arrow-ipc/28.0.0/arrow_ipc/writer/struct.IpcDataGenerator.html
Describe alternatives you've considered
Additional context
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Something I discovered while working on code base at Coralogix and in Ballista.
arrow_flight::utils::flight_data_from_arrow_batchis used as method to convert a singleRecordBatchtoFlightData.This method extracts both the dictionaries as the
RecordBatches.While convenient, if the dictionary is the shared across batches, the caller will probably send the
FlightDatafor these dictionaries over and over again over in the stream.Describe the solution you'd like
Mark the method as deprecated to encourage users to use the
IpcDataGeneratorAPI instead.https://docs.rs/arrow-ipc/28.0.0/arrow_ipc/writer/struct.IpcDataGenerator.html
Describe alternatives you've considered
Additional context