Skip to content

Synthetics Beta Packaging, Docker, Fleet, Build, and other Concerns #140

@andrewvc

Description

@andrewvc

For synthetics to reach beta (hopefully as soon as 7.11) and later GA (as soon as 7.12), we'll need to have a more robust packaging system that makes sense in the context of the wider stack.

Today, we have two things we version, the stack, and the APM agents (e.g. RUM, Java, etc.). The APM agents usually have their own versioning system independent of the stack. We can consider, for our purposes, the synthetics package to be an agent. We also need to have special dependencies present in our docker image to run synthetics, and we also need to eventually support the Elastic agent.

I think where we want to wind up here is in the following spot:

  1. The synthetics NPM package is bumped to 1.0.0-beta
  2. The heartbeat synthetics PR is merged into the main beats repo and becomes a part of regular beats distributions (right now you must use the special experimental docker image)
  3. We add all the extra synthetics RPM deps AND add the node.js to the main docker images for heartbeat and fleet agent (note, there are a bunch of additional RPM packages added to support chrome, not full X11 but many other GUI libs). See the synthetics Dockerfile for details
  4. We get rid of the custom docker images for synthetics
  5. We make the default behavior of heartbeat such that it automatically pulls the latest compatible version of elastic synthetics periodically (and by transitive dependency chromium) to ensure that the browser is evergreen (we can allow users to override this).
  6. We maintain a way for developers of this repo to build custom docker images based on their local code.
  7. We consider documenting a means of using plain heartbeat/agent (e.g. no docker) with synthetics (users install their own version of node + supporting libs).

I'd love some opinions on this strategy. Some questions I can foresee:

  • Why would we ship node.js with every fleet agent docker image? The alternative would be dynamically installing it, which AFAIK is beyond fleet's current capabilities. I'm open to this, however, it would have to work for air-gapped networks too. That said, the docker image should probably be optimized for centralized, not edge use cases IMHO.
  • Why do we need to keep updating synthetics + the browser?: Keeping a browser up to date is more secure, they have wide security surface areas and zero days happen, also, browsers are evergreen now, an old browser isn't very useful for testing. This could be disabled with a flag or env var

CC @paulb-elastic @drewpost @kuisathaverat @ruflin @mostlyjason @jahtalab @vigneshshanmugam @shahzad31 @Kerry350

Metadata

Metadata

Assignees

No one assigned

    Labels

    discussionDiscuss about API changes, enhancementsenhancementNew feature or requestv7.13.0

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions