-
Notifications
You must be signed in to change notification settings - Fork 236
Description
TestMultipleVMs_Isolated has in the past been very helpful in finding rare race-conditions due to the fact that it (unsurprisingly) spins up multiple VMs concurrently. However, it was previously limited in how many VMs it could spin up seemingly due to the use of the naive snapshotter, which appeared to become a large bottleneck when many container snapshots were being made in parallel. It currently only spins up 5 VMs each with either 5 or 3 containers.
Now that we use devmapper in our CI system, it's worth seeing if we can increase the number of VMs+containers we spin up in the test as it could give us higher chances of catching rare race conditions. If devmapper does in fact improve the situation, updating this test could serve as a primitive form of a load test, though it would still focus on making functional assertions at this time.
This may require a little extra work as while it would be great for our CI hosts to spin up something like 100 VMs in parallel if feasible, we don't want that to be the default as most developers are not going to have a host with enough RAM to start up 100 VMs.