This is a feature meta issue to implement the memory queue to output pipeline in the shipper. The scope is restricted to implementation of the memory queue and an output with no external dependencies (the console or file output for example). The disk queue, Elasticsearch/Kafka/Logstash outputs, and processors are explicitly out of scope.

This feature is considered complete when at least the following criteria are satisfied:

• A test exists to prove that data written to the shipper event gRPC interface is publishes to the output. The test should write single events and batches, including batches that are as large as the configured size of the queue to prove it does not block.
• A test exists to prove that the shipper will backpressure the producer when the queue has been filled. Ideally this means the producer will block until there is enough space in the queue. The backpressure should stop once the queue begins to drain.

The assignee of this issue is expected to create the development plan with all child issues for this feature. The following set of tasks should be included in the initial issues at a minimum:

• Implementing the event publishing RPC and have it write to the queue.
• Add queue and output sections to the shipper configuration file. The format must match the format used in agent policy output sections today.
• Create the queue and output pipeline based on the provided configuration. Allow the configuration to be refreshed.
• Creation of an integration test suite for the shipper process.

Important milestones:

• Adapt the memory queue to accept shipper types (Make the memory queue work with types other than publisher.Event beats#31307)
• Create a memory queue in the shipper binary and propagate input events through it
• Create a test output that can confirm events received from the queue
• Create an integration test that invokes the gRPC publishing interface and verifies its handling via the test output