With upgrade in mind we need to align structure of where things go to make scenarios smooth and possible.

Proposed structure

/data
  /v7.9.1-abc123d
     /downloads
     /install
     /run
     elastic-agent
  /v7.9.2-dba324e
     /downloads
     /install
     /run
     elastic-agent
     elastic-agent.yml
     fleet.yml
     action_store.yml
  /logs
elastic-agent
elastic-agent.yml
fleet.yml
action_store.yml

v7.9.1-ab123d is a v7.9.1 semver version where the rest of the string is a hash of a version which can contain suffixes like SNAPSHOT, BC...
each version contains its own binary and dependent binaries in download/install directories

in this example /v7.9.2-dba324e is an older version which contains not only binary but snapshot of config files and actionstore

logs were moved from version level to root level of the structure. this is so monitoring wont drop any events which might be unprocessed or generated in between upgrade steps.

run is used during runtime to store sockets etc.

elastic-agent at root level is a symlink to a currently active version, any service manager should point to this file as executable. this gets updated on update/rollback

elastic-agent.yml, fleet.yml and action_store.yml are config/state files which are used by active version of an agent. during upgrade process these are copied to version folder overwriting any previously generated config files if any.
agent copies these files on start from its versioned directory if it contains any and removes them to avoid future overwrite.

older versioned folders are removed after grace period without beats in FAILED state together with prev symlink (if used). after this point rollback wont be possible.

cc @ph @blakerouse