Proposal

The promql testing framework currently treats annotations as follows:

• eval and eval_ordered require no annotation.
• eval_warn requires one or more warn annotations, but no info annotation.
• eval_info requires one or more info annotations, but no warn annotation.

This has the following shortcomings:

• It is impossible to use eval_ordered for an expression that returns one or more annotations.
• It is impossible to test for a combination of warn and info annotations.
• It is impossible to test for specific annotations (in contrast to what is possible for errors via expected_fail_message).
• It is impossible to write a test that doesn't care about annotations at all.

Clearly, we cannot just increase the multiplicity of eval_... keywords to cover all permutations (eval_warn_info, eval_ordered_warn etc.). Furthermore, we cannot easily extend the syntactical approach of expected_fail_message because the latter replaces the result section (but annotations are coming in addition to a result).