APM Server now has an autoscaling Elasticsearch output (specialized to the APM data model). We should explore porting their implementation into the shipper with the goal of eliminating the need to manually tune the number of workers and the output batch size.

The initial APM autoscaling implementation was done in elastic/apm-server#9393 and linked to the following tracking issues:

• https://github.com/elastic/apm-dev/issues/820
  • Autoscale number of modelindexers to increase throughput and ensure full resource usage apm-server#9181

When we begin this work we should sync with the APM Server team to come up to speed on the latest state of their autoscaling implementation.