Describe the bug

Data Prepper end-to-end acknowledgements have expiration times. These times are set to 10 minutes by default.

The current implementation of DynamoDB streams will hold a single AcknowledgementSet until the whole shard is processed. A DynamoDB shard can remain open for up to four hours with the source coordination. So it is quite easy for the AcknowledgementSet to take up to four hours. This means that the acknowledgement set will expire.

Expected behavior

Acknowledgement sets will not expire while processing a shard.

Proposed Solution

Update the DynamoDB source to increase the AcknowledgementSet expiryTime by increasing for 10 minutes.

This can be done by updating the ProgressCheck interface and implementing this behavior to change the expiryTime.

public interface ProgressCheck {
    /**
     * Returns the pending ratio
     *
     * @return returns the ratio of pending to the total acknowledgements
     * @since 2.6
     */
    Double getRatio();

    /**
     * Increases the expiry time of the acknowledgement set by the given duration
     *
     * @param additionalTime additional time to be added to the expiry time
     * @since 2.11
     */
    void increaseExpiry(Duration additionalTime);
}

After this, subscribe the dynamodb source to progress checks and use the callback to increase the expiry.

Environment (please complete the following information):

Data Prepper 2.10.1

Additional context

A fuller solution could be provided by #4764.