The agent E2E tests have not been stable for a long time, which has made them ineffective as a quality check for agent PR and releases. The E2E tests no longer act as a gate for agent releases for this reason.
Fixing all of the flaky E2E tests is too large of a task for a single issue, but to ensure we get some value from end to end tests we can restore the most valuable test suites and make them reliable on a reduced set of platforms as a start point.
The scope of this issue is to reduce the set of E2E tests triggered by agent builds to just the Fleet Mode and Upgrade Agent test suites and make them reliable when run on Ubuntu and Windows. The E2E test framework already supports custom test run profiles based on the repository that triggers them, see the existing profile for the agent here.
The work that needs to be done is also tracked in the following issue created by the obs-robots team.
The agent E2E tests have not been stable for a long time, which has made them ineffective as a quality check for agent PR and releases. The E2E tests no longer act as a gate for agent releases for this reason.
Fixing all of the flaky E2E tests is too large of a task for a single issue, but to ensure we get some value from end to end tests we can restore the most valuable test suites and make them reliable on a reduced set of platforms as a start point.
The scope of this issue is to reduce the set of E2E tests triggered by agent builds to just the Fleet Mode and Upgrade Agent test suites and make them reliable when run on Ubuntu and Windows. The E2E test framework already supports custom test run profiles based on the repository that triggers them, see the existing profile for the agent here.
The work that needs to be done is also tracked in the following issue created by the obs-robots team.