bug: Unable to forward SNS message to SQS: An error occurred (AWS.SimpleQueueService.EmptyBatchRequest) when calling the SendMessageBatch #7662
Description
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
I have a setup of a SNS fifo topic and multiple SQS fifo queues subscribing to this topic with filter policies.
I'm getting this error from time to time and then SNS/SQS stops working, to make it work again I need to restart the localstack container.
The error is thrown from this line.
https://github.com/localstack/localstack/blob/master/localstack/services/sns/publisher.py#L363
So I did a little change to print what's in the entries object
print("------------------------------")
print(entries)
print("------------------------------")
response = sqs_client.send_message_batch(QueueUrl=queue_url, Entries=entries)And as you can see it is empty, and then SQS fails as expected because the entries cannot be empty.
I guess that because some messages are filtered by the filter policy
2023-02-11 10:25:27 2023-02-11T13:25:27.353 INFO --- [ asgi_gw_1] localstack.request.aws : AWS sqs.ReceiveMessage => 200
2023-02-11 10:25:33 ------------------------------
2023-02-11 10:25:33 []
2023-02-11 10:25:33 ------------------------------
2023-02-11 10:25:33 2023-02-11T13:25:32.645 DEBUG --- [ asgi_gw_4] l.services.sns.publisher : Topic 'arn:aws:sns:us-east-1:000000000000:<redacted>-events.fifo' batch publishing 1 messages to subscribed 'arn:aws:sqs:us-east-1:000000000000:<redacted>-events.fifo' with protocol 'sqs' (subscription 'arn:aws:sns:us-east-1:000000000000:<redacted>-events.fifo:f1ef8bc0-f5ed-4da8-b0f8-3b103ca9cb54')
2023-02-11 10:25:33 2023-02-11T13:25:32.645 DEBUG --- [ asgi_gw_4] l.services.sns.publisher : No messages match filter policy, not publishing batch from topic 'arn:aws:sns:us-east-1:000000000000:<redacted>-events.fifo' to subscription 'arn:aws:sns:us-east-1:000000000000:<redacted>-events.fifo:71de2566-97e6-48b8-a222-54c9adadcf96'
2023-02-11 10:25:33 2023-02-11T13:25:32.645 INFO --- [ asgi_gw_4] localstack.request.aws : AWS sns.PublishBatch => 200
2023-02-11 10:25:33 2023-02-11T13:25:32.656 INFO --- [ sns_pub_0] l.services.sns.publisher : Unable to forward SNS message to SQS: An error occurred (AWS.SimpleQueueService.EmptyBatchRequest) when calling the SendMessageBatch operation: Unknown Traceback (most recent call last):
2023-02-11 10:25:33 File "/opt/code/localstack/localstack/services/sns/publisher.py", line 366, in _publish
2023-02-11 10:25:33 response = sqs_client.send_message_batch(QueueUrl=queue_url, Entries=entries)
2023-02-11 10:25:33 File "/opt/code/localstack/.venv/lib/python3.10/site-packages/botocore/client.py", line 530, in _api_call
2023-02-11 10:25:33 return self._make_api_call(operation_name, kwargs)
2023-02-11 10:25:33 File "/opt/code/localstack/.venv/lib/python3.10/site-packages/botocore/client.py", line 960, in _make_api_call
2023-02-11 10:25:33 raise error_class(parsed_response, operation_name)
2023-02-11 10:25:33 botocore.errorfactory.EmptyBatchRequest: An error occurred (AWS.SimpleQueueService.EmptyBatchRequest) when calling the SendMessageBatch operation: Unknown
2023-02-11 10:25:33
Expected Behavior
When batch publishing from SNS topic to SQS subscription, first it needs to check if the entries object is not empty.
How are you starting LocalStack?
With a docker-compose file
Steps To Reproduce
docker-compose.yml
version: "3.8"
services:
localstack:
container_name: droz-localstack
image: localstack/localstack
ports:
- "4566:4566" # LocalStack Gateway
environment:
- DEBUG=1
- SERVICES=sqs,sns,dynamodb
- DATA_DIR=/tmp/localstack/data
- AWS_DEFAULT_REGION=us-east-1
volumes:
- ./.localstack:/var/lib/localstack
volumes:
localstack:Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)
TBD
Environment
- OS: macOs
- LocalStack: latestAnything else?
No response