Skip to content

Reduces Dockerfile image size by 46%#6520

Merged
dlvenable merged 1 commit intoopensearch-project:mainfrom
dlvenable:docker-size-reduction
Feb 18, 2026
Merged

Reduces Dockerfile image size by 46%#6520
dlvenable merged 1 commit intoopensearch-project:mainfrom
dlvenable:docker-size-reduction

Conversation

@dlvenable
Copy link
Copy Markdown
Member

@dlvenable dlvenable commented Feb 16, 2026

Description

Reduces the Docker image size by 46%. This also removes seven layers. This has the following optimizations:

  • Run chown on COPY. The previous approach resulted in duplicate data from copies.
  • Consolidated layers that could be combined. This probably doesn't reduce the overall size, but helps with pull time through reduced layers.
  • Install, use, and remove shadow-utils in a single layer to keep the project out of the image.
  • Remove dnf caches by running dnf clean and rm -rf /var/cache/dnf

Comparisons

Data Prepper 2.13.0 layers - The latest release

docker history opensearchproject/data-prepper:2.13.0 --human

IMAGE          CREATED        CREATED BY                                      SIZE      COMMENT
7fa42f9657cc   2 months ago   CMD ["bin/data-prepper"]                        0B        buildkit.dockerfile.v0
<missing>      2 months ago   WORKDIR /usr/share/data-prepper                 0B        buildkit.dockerfile.v0
<missing>      2 months ago   USER data_prepper                               0B        buildkit.dockerfile.v0
<missing>      2 months ago   RUN |4 PIPELINE_FILEPATH=/usr/share/data-pre…   334MB     buildkit.dockerfile.v0
<missing>      2 months ago   COPY default-keystore.p12 /usr/share/data-pr…   2.47kB    buildkit.dockerfile.v0
<missing>      2 months ago   COPY default-data-prepper-config.yaml /usr/s…   245B      buildkit.dockerfile.v0
<missing>      2 months ago   RUN |4 PIPELINE_FILEPATH=/usr/share/data-pre…   334MB     buildkit.dockerfile.v0
<missing>      2 months ago   ADD opensearch-data-prepper-2.13.0-linux-x64…   334MB     buildkit.dockerfile.v0
<missing>      2 months ago   RUN |4 PIPELINE_FILEPATH=/usr/share/data-pre…   0B        buildkit.dockerfile.v0
<missing>      2 months ago   RUN |4 PIPELINE_FILEPATH=/usr/share/data-pre…   357MB     buildkit.dockerfile.v0
<missing>      2 months ago   ADD adoptium.repo /etc/yum.repos.d/adoptium.…   192B      buildkit.dockerfile.v0
<missing>      2 months ago   RUN |4 PIPELINE_FILEPATH=/usr/share/data-pre…   2.86kB    buildkit.dockerfile.v0
<missing>      2 months ago   RUN |4 PIPELINE_FILEPATH=/usr/share/data-pre…   195kB     buildkit.dockerfile.v0
<missing>      2 months ago   RUN |4 PIPELINE_FILEPATH=/usr/share/data-pre…   7.94MB    buildkit.dockerfile.v0
<missing>      2 months ago   RUN |4 PIPELINE_FILEPATH=/usr/share/data-pre…   114MB     buildkit.dockerfile.v0
<missing>      2 months ago   ENV ENV_PIPELINE_FILEPATH=/usr/share/data-pr…   0B        buildkit.dockerfile.v0
<missing>      2 months ago   ENV ENV_CONFIG_FILEPATH=/usr/share/data-prep…   0B        buildkit.dockerfile.v0
<missing>      2 months ago   ENV DATA_PREPPER_PATH=/usr/share/data-prepper   0B        buildkit.dockerfile.v0
<missing>      2 months ago   ARG ARCHIVE_FILE_UNPACKED=opensearch-data-pr…   0B        buildkit.dockerfile.v0
<missing>      2 months ago   ARG ARCHIVE_FILE=opensearch-data-prepper-2.1…   0B        buildkit.dockerfile.v0
<missing>      2 months ago   ARG CONFIG_FILEPATH=/usr/share/data-prepper/…   0B        buildkit.dockerfile.v0
<missing>      2 months ago   ARG PIPELINE_FILEPATH=/usr/share/data-preppe…   0B        buildkit.dockerfile.v0
<missing>      3 months ago   CMD ["/bin/bash"]                               0B        buildkit.dockerfile.v0
<missing>      3 months ago   ADD al2023-container-raw-2023.9.20251117.1-a…   148MB     buildkit.dockerfile.v0

Data Prepper 2.14.0-SNAPSHOT layers - The local changes

docker history opensearch-data-prepper:2.14.0-SNAPSHOT --human

IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT
25d9b6aeef14   19 minutes ago   CMD ["bin/data-prepper"]                        0B        buildkit.dockerfile.v0
<missing>      19 minutes ago   WORKDIR /usr/share/data-prepper                 0B        buildkit.dockerfile.v0
<missing>      19 minutes ago   USER data_prepper                               0B        buildkit.dockerfile.v0
<missing>      19 minutes ago   COPY --chown=1000:1000 default-keystore.p12 …   2.47kB    buildkit.dockerfile.v0
<missing>      19 minutes ago   COPY --chown=1000:1000 default-data-prepper-…   245B      buildkit.dockerfile.v0
<missing>      19 minutes ago   COPY --chown=1000:1000 /tmp/data-prepper /us…   335MB     buildkit.dockerfile.v0
<missing>      19 minutes ago   RUN |2 PIPELINE_FILEPATH=/usr/share/data-pre…   0B        buildkit.dockerfile.v0
<missing>      47 minutes ago   RUN |2 PIPELINE_FILEPATH=/usr/share/data-pre…   368MB     buildkit.dockerfile.v0
<missing>      48 minutes ago   ADD adoptium.repo /etc/yum.repos.d/adoptium.…   192B      buildkit.dockerfile.v0
<missing>      48 minutes ago   RUN |2 PIPELINE_FILEPATH=/usr/share/data-pre…   3.54MB    buildkit.dockerfile.v0
<missing>      48 minutes ago   ENV ENV_PIPELINE_FILEPATH=/usr/share/data-pr…   0B        buildkit.dockerfile.v0
<missing>      48 minutes ago   ENV ENV_CONFIG_FILEPATH=/usr/share/data-prep…   0B        buildkit.dockerfile.v0
<missing>      48 minutes ago   ENV DATA_PREPPER_PATH=/usr/share/data-prepper   0B        buildkit.dockerfile.v0
<missing>      48 minutes ago   ARG CONFIG_FILEPATH=/usr/share/data-prepper/…   0B        buildkit.dockerfile.v0
<missing>      48 minutes ago   ARG PIPELINE_FILEPATH=/usr/share/data-preppe…   0B        buildkit.dockerfile.v0
<missing>      13 days ago      CMD ["/bin/bash"]                               0B        buildkit.dockerfile.v0
<missing>      13 days ago      ADD al2023-container-raw-2023.10.20260202.2-…   182MB     buildkit.dockerfile.v0

Final image sizes

docker image ls | grep data-prepper

opensearch-data-prepper                  2.14.0-SNAPSHOT   25d9b6aeef14   20 minutes ago   888MB
opensearchproject/data-prepper           2.13.0            7fa42f9657cc   2 months ago     1.63GB

Issues Resolved

Resolves #3356

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dlvenable
Reducing Dockerfile image size greatly by consolidating layers, runni…
13299fa 
…ng chown on copy to avoid duplicate data, running dnf clean, and removing dnf cache.

Signed-off-by: David Venable <dlv@amazon.com>
@dlvenable dlvenable changed the title Reducing Dockerfile image size greatly by consolidating layers, runni… Reducing Dockerfile image size by 46% Feb 16, 2026
@dlvenable dlvenable changed the title Reducing Dockerfile image size by 46% Reduces Dockerfile image size by 46% Feb 16, 2026
@dlvenable dlvenable merged commit 22177f0 into opensearch-project:main Feb 18, 2026
74 of 76 checks passed
@dlvenable dlvenable deleted the docker-size-reduction branch March 4, 2026 16:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@oeyh oeyh oeyh approved these changes

@sb2k16 sb2k16 sb2k16 approved these changes

@engechas engechas Awaiting requested review from engechas engechas is a code owner

@san81 san81 Awaiting requested review from san81 san81 is a code owner

@srikanthjg srikanthjg Awaiting requested review from srikanthjg srikanthjg is a code owner

@graytaylor0 graytaylor0 Awaiting requested review from graytaylor0 graytaylor0 is a code owner

@dinujoh dinujoh Awaiting requested review from dinujoh dinujoh is a code owner

@kkondaka kkondaka Awaiting requested review from kkondaka kkondaka is a code owner

@KarstenSchnitter KarstenSchnitter Awaiting requested review from KarstenSchnitter KarstenSchnitter is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Reduce the Data Prepper tar.gz and Docker image sizes

3 participants

@dlvenable @oeyh @sb2k16