Request

The current SQS listener job (in log-ingestor) only uses one coroutine to listen to the SQS. As requested by our user, the ingestion throughput expected would be 15k SQS messages per minute. To adapt this throughput, we need more coroutines to process SQS messages concurrently inside one SQS listener job.

Possible implementation

• Allow users to specify the number of coroutines per job.
• All coroutines execute the same ingestion task, while the tasks are the multi-source (sender) of the same buffer for compression.