It's quite disruptive to add options to everything, given that Prometheus itself doesn't need it.

Or to be precise: There are plans to make the eval_... commands better suited to test various annotation scenarios. I would strongly prefer to add more commands (or change the existing ones) so that tests can express things like "I don't care about annotations" or "I want both info and warn" or even "I want this very specific annotation".

See the discussion starting here. This would get rid of eval_info and eval_warn altogether, replacing them by some other parametrization to specify expected infos and warnings, and that should also help your use case here. I'll try to file a proper issue for it later today.

In any case (as also discussed there), the short-term plan is to make eval_ordered ignore annotations. Maybe that's something you could piggyback on until we have the "proper" solution in place.

See #15794 for the "proper" solution.

#15795 will at least allow eval_ordered to circumvent annotation checks. (It doesn't work under all circumstances, but maybe it already helps.)

I think #15794 looks very promising by now. Let's get #15794 done ASAP, which should also solve the problem this PR tries to address. In turn, let's close this to not create the additional complexity in the code that we would then remove again.