We have a rather complex set of situations where a test is either skipped or expected to fail. As such we can't use the standard #[should_panic] or #[ignored] as we need to make that decision at runtime based on the device we're running on.

This can cause problems where the developer is trying to fix a particular problem, runs the tests, sees that they all pass, not realizing that the test they're trying to measure against isn't actually running or has an expected fail on it.

We should find some way to express if something is skipped.

One possibility is to use libtest-mimic to allow us to add a "kind" to tests which is then visible to cargo-nextest or cargo test. We likely would need our own macro for tests in that case and would have interesting ramifications for wasm testing via wasm-pack.