Skip to content

[8.19] (backport #9386) Enhancement/5235 upgrade cleans up downloads and extracted agent#10037

Merged
kaanyalti merged 2 commits into8.19from
mergify/bp/8.19/pr-9386
Sep 24, 2025
Merged

[8.19] (backport #9386) Enhancement/5235 upgrade cleans up downloads and extracted agent#10037
kaanyalti merged 2 commits into8.19from
mergify/bp/8.19/pr-9386

Conversation

@mergify
Copy link
Copy Markdown
Contributor

@mergify mergify bot commented Sep 18, 2025

PR 5/6

  • Enhancement

What does this PR do?

  • Updates the download step to return path after download succeeds even in error cases
  • Adds deferred cleanup process in the upgrade function to cleanup download and extracted agent versioned home paths
  • Updates error handling sequence in the upgrade function to add download and versioned home paths in the cleanup list
  • Adds relevant tests

Why is it important?

  • Currently http and fs downloaders cleanup if they fail however if there is any other error (verifier errors) in the download process, the agent does not cleanup the downloaded artifacts
  • The upgrade process reverts the symlink changes and cleans up the new versioned home only after the symlink switch happens. If anything in between the downloaders and symlink switch fails, the agent does not clean up anything.

Checklist

  • I have read and understood the pull request guidelines of this project.
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • [ ] I have added an entry in ./changelog/fragments using the changelog tool
  • [ ] I have added an integration test or an E2E test

Disruptive User Impact

none

How to test this PR locally

  • Run the upgrade_test tests
  • Build the agent for windows, mac and linux
  • Try to upgrade from one version to another without sha512 and force verification failure and validate that the downloads directory is cleaned up.
  • The way I triggered the second cleanup scenario was by filling up the disk space just enough so that the agent can partially extract the downloaded archive but fails midway. Then I validated that the versioned home for the new agent is removed along with the downloads in the initial agent's versioned home directory.

Related issues

This is an automatic backport of pull request #9386 done by [Mergify](https://mergify.com).

@mergify
Enhancement/5235 upgrade cleans up downloads and extracted agent (#9386)
64bf514 
* enhancement(5235): added comments describing updates in error handling. returning path in error cases if the download is completed

* enhancement(5235): added wrapper var for verifier
constructor in download step to improve testability

* enhancement(5235): added test for downloadArtifact
verifier errors

* enhancement(5235): addded cleanupPaths slice. added deferred call to remove paths in the cleanupPaths slice

* enhancement(5235): add archive and hash path to cleanup paths after artifact download

* enhancement(5235): update error handling after unpack. add new versioned home to cleanup paths

enhancement(5235): move newHome declaration

* enhancement(5235): added upgrade cleanup test

* enhancement(5235): added changelog fragment

enhancement(5235): updated changelog fragment

* enhancement(5235): fix errors in step_download after rebase

* enhancement(5235): fix error in step_download_test after rebase

* enhancement(5235): refactor disk space error check and path cleanup into one deferred call

* enhancement(5235): refactored verifier factory and updated relevant download tests

* enhancement(5235): removed separate test for cleanup, added cleanup checks in error handling test

* enhancement(5235): changed function name in verifier

* enhancement(5235): using AddHashExtension instead of hard coded hash extension string in upgrader

(cherry picked from commit ab23962)

# Conflicts:
#	internal/pkg/agent/application/upgrade/upgrade.go
#	internal/pkg/agent/application/upgrade/upgrade_test.go
@mergify mergify bot requested a review from a team as a code owner September 18, 2025 16:56
@mergify mergify bot added backport conflicts There is a conflict in the backported pull request labels Sep 18, 2025
@mergify mergify bot requested review from pchila and ycombinator and removed request for a team September 18, 2025 16:56
@mergify
Copy link
Copy Markdown
Contributor Author

mergify bot commented Sep 18, 2025

Cherry-pick of ab23962 has failed:

On branch mergify/bp/8.19/pr-9386
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit ab23962a0.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1755268130-cleanup-downloads-directory-and-the-new-versioned-home-if-upgrade-fails.yaml
	modified:   internal/pkg/agent/application/upgrade/artifact/download/verifier.go
	modified:   internal/pkg/agent/application/upgrade/step_download.go
	modified:   internal/pkg/agent/application/upgrade/step_download_test.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   internal/pkg/agent/application/upgrade/upgrade.go
	both modified:   internal/pkg/agent/application/upgrade/upgrade_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@mergify mergify bot mentioned this pull request Sep 18, 2025
Merged
4 tasks
@github-actions github-actions bot added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Sep 18, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Sep 18, 2025

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Sep 18, 2025 •
edited
Loading

💔 Build Failed

Failed CI Steps

History

cc @kaanyalti

@mergify
Copy link
Copy Markdown
Contributor Author

mergify bot commented Sep 22, 2025

This pull request has not been merged yet. Could you please review and merge it @kaanyalti? 🙏

@ebeahan
Copy link
Copy Markdown
Member

ebeahan commented Sep 23, 2025

@kaanyalti can you take a look at this backport?

@kaanyalti kaanyalti removed the conflicts There is a conflict in the backported pull request label Sep 24, 2025
@elastic-sonarqube
Copy link
Copy Markdown

elastic-sonarqube bot commented Sep 24, 2025

Quality Gate passed Quality Gate passed

Issues
2 New issues
1 Fixed issue
0 Accepted issues

Measures
0 Security Hotspots
90.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

@kaanyalti kaanyalti merged commit cde47ae into 8.19 Sep 24, 2025
20 checks passed
@kaanyalti kaanyalti deleted the mergify/bp/8.19/pr-9386 branch September 24, 2025 18:33
@Y-o-Z
Copy link
Copy Markdown

Y-o-Z commented Apr 13, 2026

Hi @kaanyalti, nice to e-meet you :)

The way I read this GitHub PR, agents of version 8.19 should now be able to clean downloaded files if the agent update fails for some reason (e.g. insufficient disk space).

I just tested this on Rocky 8.8. I tried to update a 8.19.11 Elastic Agent to 8.19.12.

Screenshot 2026-04-13 at 10 07 41

I filled up the installation directory except for +- 300 MB and started the Elastic Agent upgrade process via Fleet. As excepted, the upgrade eventually failed with error_msg: insufficient disk space

However, the downloaded files remain on the host and /opt/ remains at 100% disk usage.

Do I misunderstand what you tried to achieve here or is your enhancement not working correctly?

I'm happy to provide more details if needed, as this is a feature that we would really like to have.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ycombinator ycombinator Awaiting requested review from ycombinator ycombinator is a code owner automatically assigned from elastic/elastic-agent-control-plane

@pchila pchila Awaiting requested review from pchila pchila is a code owner automatically assigned from elastic/elastic-agent-control-plane

1 more reviewer

@kaanyalti kaanyalti kaanyalti approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@kaanyalti kaanyalti

Labels

backport Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@elasticmachine @ebeahan @Y-o-Z @kaanyalti