In Beats, vending of events from the queue and acknowledgment of those events by the outputs is done in "batches," arrays of consecutive events that are sent together over the network, and are removed from the queue as a single unit when complete.

In the agent shipper, outputs no longer use the batch abstraction -- the boundary of a network "batch" can happen anywhere based on the configuration of the library (for example, go-elasticsearch can send batches based on a configured byte limit). Acknowledgments may be received per-event and need to be propagated back to the queue so completed messages can be freed from memory/disk.

Currently this mismatch between the shipper and the beats queue is handled by wrapping fixed-size queue batches in a helper object that listens to callbacks and tracks how many events from that batch are still outstanding. This decouples the go-elasticsearch workers from the batch boundaries in the originating queue. Instead, we need the core queue API to support reading single events, so we should move the assembly of those events into batches into an auxiliary helper that can preserve the same behavior in Beats while allowing the shipper to ignore the batch interface and propagate event acknowledgments directly.