Allow native shuffle batch size to be configured separately from comet default batch size #1187
Description
What is the problem the feature request solves?
I get better performance with larger batch sizes in shuffle writer (at the expense of using more memory). It would be nice to make this configurable separately from the current batch size config.
Describe the potential solution
No response
Additional context
No response