The shipper currently defines a configuration file with the following format: https://github.com/elastic/elastic-agent-shipper/blob/main/elastic-agent-shipper.yml

This configuration file was implemented before the design decisions needed to integrate the shipper into the agent were finalized. We need to update it to match the final design decisions. Specifically it was decided that:

1. The agent will provide the shipper with a copy of the entire agent policy, so that it can parse out the fields it needs to implement features that depend on input configuration like processing. Provide the relevant subset of the agent policy to the shipper as its configuration elastic-agent#617 (comment)
2. The shipper will be enabled on a per output basis by including a shipper: sub-object in the configuration for each agent output. All new shipper specific configuration should be nested under this shipper: configuration key, including queue configuration. Define a feature flag for enabling the shipper in the agent policy, defaulting to false. elastic-agent#217 (comment). For example:

outputs:
  default:
    type: elasticsearch
    hosts: https://localhost:9200
    shipper:
       enabled: true
       queue: {}

3. The shipper's control protocol connection will receive both input and output units. The output units will match the output section of an agent policy. There will be one input unit per connected component (process or Beat) configuring the gRPC connection between that component and the shipper, including the mTLS certificate to use.

These changes are implemented in elastic/elastic-agent#1527 which adds support for the shipper to agent.

The scope of this issue is to incorporate the new shipper configuration model into the shipper, and test that it is compatible with with the agent changes in elastic/elastic-agent#1527.

Key cases to test are:

1. Multiple components (Beat processes) can successfully connect to a single shipper instance and publish events.
2. Multiple outputs can be configured in an agent policy, each individually configuring a data shipper. Ensure that each individual shipper process is started and configured correctly.