As I understand it:

• create_virtual_hardware.sh and destroy_virtual_hardware.sh create and destroy devices that make an ordinary system look more like a Gimlet for Omicron.
• Omicron itself creates other devices that depend on these devices. These dependencies prevent the lower-level devices from being destroyed. Concretely, destroy_virtual_hardware.sh fails if Omicron hasn't been uninstalled -- see omicron-package uninstall leaves a bunch of networking stuff around #1212.
• The intended invariants are:
  • Omicron only runs after a successful call to create_virtual_hardware.sh and before a call to destroy_virtual_hardware.sh, such that Omicron can always assume these devices are around
  • destroy_virtual_hardware.sh assumes that Omicron has been uninstalled.

If you don't already know the specific devices created by each step and how they depend on each other, then if you get this wrong (or just run into #1212), it's hard to figure out what's wrong. It'd be nice if destroy_virtual_hardware.sh could check its own preconditions (i.e., that Omicron has been uninstalled) or else produce a clearer error message when it fails because Omicron hasn't been uninstalled. Maybe if it gets "device busy" from dladm, it could prompt the user with something like "have you uninstalled Omicron?" [and maybe "see also issue #1212"]