(lambda-event-sources): on_failure parameter is not supported for SelfManagedKafkaEventSource #19917
Description
Describe the bug
As per (https://docs.aws.amazon.com/cdk/api/v1/python/aws_cdk.aws_lambda_event_sources/SelfManagedKafkaEventSource.html
https://docs.aws.amazon.com/cdk/api/v2/python/aws_cdk.aws_lambda_event_sources/SelfManagedKafkaEventSource.html) we can specify on_failure parameter in the SelfManagedKafkaEventSource.
But the CloudFormation stack deployment fails with below error:
"Resource handler returned message: "Invalid request provided: Unsupported 'destinationConfig' parameter for given event source mapping type. (Service: Lambda, Status Code: 400)"
Upon checking the CFN documentation (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-eventsourcemapping.html#cfn-lambda-eventsourcemapping-destinationconfig) for AWS::Lambda::EventSourceMapping resource, adding destinationConfig is only supported for 'Stream' event sources, which are DynamoDB and Kinesis.
I spun up a test Kafka cluster and then using CDK I created the lambda function that has self-managed Apache Kafka event source.
mylambda = lambda_.Function(self, "MyLambda",
code=lambda_.Code.from_asset("lambda"),
handler="index.handler",
runtime=lambda_.Runtime.NODEJS_12_X,
role=role
)
mylambda.add_event_source(SelfManagedKafkaEventSource(
bootstrap_servers=bootstrap_servers,
topic=topic,
batch_size=100, # default
starting_position=lambda_.StartingPosition.LATEST,
vpc=vpc,
vpc_subnets=ec2.SubnetSelection(
subnet_type=ec2.SubnetType.PRIVATE
),
security_group=security_group
on_failure=SqsDlq(dead_letter_queue)
))
The cdk synth command ran successfully.
After running the cdk deploy command, the corresponding CFN stack failed with below error:
Resource handler returned message: "Invalid request provided: Unsupported 'destinationConfig' parameter for given event source mapping type. (Service: Lambda, Status Code: 400)
This implies that on_failure parameter is not supported for SelfManagedKafkaEventSource.
Also, on AWS console, if you try adding the Apache Kafka event source, you will not find the On-failure destination option.
But you can see the On-failure destination option for DynamoDB and Kinesis event source.
Expected Behavior
For AWS::Lambda::EventSourceMapping resource, adding DestinationConfig is only supported for 'Stream' event sources, which are DynamoDB and Kinesis.
So, this option should be removed from SelfManagedKafkaEventSource.
Current Behavior
The cdk synth command ran successfully.
After running the cdk deploy command, the corresponding CFN stack failed with below error:
Resource handler returned message: "Invalid request provided: Unsupported 'destinationConfig' parameter for given event source mapping type. (Service: Lambda, Status Code: 400)
Reproduction Steps
mylambda = lambda_.Function(self, "MyLambda",
code=lambda_.Code.from_asset("lambda"),
handler="index.handler",
runtime=lambda_.Runtime.NODEJS_12_X,
role=role
)
mylambda.add_event_source(SelfManagedKafkaEventSource(
bootstrap_servers=bootstrap_servers,
topic=topic,
batch_size=100, # default
starting_position=lambda_.StartingPosition.LATEST,
vpc=vpc,
vpc_subnets=ec2.SubnetSelection(
subnet_type=ec2.SubnetType.PRIVATE
),
security_group=security_group
on_failure=SqsDlq(dead_letter_queue)
))
Possible Solution
No response
Additional Information/Context
No response
CDK CLI Version
1.152.0 (build 9487b39)
Framework Version
No response
Node.js Version
v16.13.0
OS
Mac
Language
Python
Language Version
No response
Other information
No response