Is your feature request related to a problem? Please describe.

Data Prepper currently has both end-to-end tests and a set of smoke tests. The end-to-end tests cover more cases than the smoke tests.

The primary advantage of the smoke tests is that they use the actual Docker image or tar.gz. We can do the same thing with the end-to-end tests. Then we'd only have one set of tests to maintain.

Describe the solution you'd like

Update the end-to-end tests to build their Docker image from the actual release Docker image instead of making a custom Docker image.

Additionally, support building the end-to-end tests using the tar.gz releases.

Describe alternatives you've considered (Optional)

Continue to have these two tests split. But, this ends up requiring more maintenance to ensure that both tests run.

Additional context

The smoke tests are not running per #2579. As I looked into these failures, I realized that the end-to-end tests are nearly what we need for smoke testing.

Tasks

• Update end-to-end tests to run from the released Docker image #3566
• Make end-to-end tests configurable such that they can run from configurable Docker images #3567
• Update release workflow to use end-to-end tests with pre-released Docker image #3568