Support SQS as a Source #1049
Description
log-pipeline:
source:
sqs:
queues:
- url: "https://sqs.us-east-1.amazonaws.com/myQueue1"
polling_frequency: 5m
batch_size: 10
number_of_threads: 2
- urls: "https://sqs.us-east-1.amazonaws.com/myQueue2"
polling_frequency: 1m
batch_size: 10
number_of_threads: 3
The data from the queue will be sent to the buffer as:
{
"message" : "{messageBody}"
}
It should also include queue metadata:
{
"queueUrl" : "https://sqs.us-east-1.amazonaws.com/myQueue1",
"sentTimestamp" : 1720679777