Functions like Test::More’s ok() automatically generate diagnostic
messages pointing to the caller’s location when a test fails. It works
fine in simple cases where Test::More is used directly in a .t file, but
it stops working here because of the DSL’s design.

As a result, test failures were always traced to some line in either
t/TestUtil/DSL.pm or t/TestUtil/DSL/Compiler.pm, which makes no sense.

In order to remedy this, we need to augment the AST with context
information about the place where scenario blocks and expect/forbid
keywords are used. Then, at execution time, we employ some rather
inelegant hacks in order to print the appropriate context information in
lieu of Test::More’s automatically generated context information should
a test fail.

Some refactoring was needed in order to make sure that todo tests were
handled appropriately. The previous implementation did not call
todo_start() at the right moment. Because of that, a test marked todo
which did pass was not reported at the end of the test harness’s output.

An issue, zonemaster/zonemaster#1401, proposes to “[u]pdate a test
scenario with requirements for message arguments”.

While as of writing this commit for the first time (2025-07-30), no
scenario has been updated yet, the tooling is implemented.

Message arguments can be tested as it is specified in the DSL’s
documentation. If such a test fails, we try our best to print as helpful
a message as possible: the test prints what exactly it was looking for
and which messages it has examined.

This debugging output could be improved by explaining why a given
message has failed a match, but the current level of detail is hopefully
plenty enough for debugging purposes.

Below is a sample TAP output detailing a failure of such a test. Long
lines are folded in order not to exceed 78 columns (the offendings lines
end with a backslash):

    t/Test-basic02.t .. 10/?
        #   Failed test 'Messages of tag 'B02_AUTH_RESPONSE_SOA' exist with\
        #   specified arguments'
        #   at t/Test-basic02.t line 112.
        # Looked for a message whose tag is 'B02_AUTH_RESPONSE_SOA'
        #   where 'domain' equals 'mixed-1.basic02.xa'
        #     and 'ns_list' equals 'ns1.mixed-1.basic02.xa/127.12.2.31;\
        #     ns1.mixed-1.basic02.xa/fda1:b2:c3:0:127:12:2:31'
        #     and contains no argument other than those listed above
        # Here are all messages that unsuccessfully matched:
        #   B02_AUTH_RESPONSE_SOA domain="mixed-1.basic02.xa"; \
        #     ns_list=ns1.mixed-1.basic02.xa/127.12.2.31;\
        #     ns1.mixed-1.basic02.xa/fda1:b2:c3:0:127:12:2:31
    #   Failed scenario 'MIXED-1'
    #   at t/Test-basic02.t line 117.
    # Looks like you failed 1 test of 26.
    t/Test-basic02.t .. Dubious, test returned 1 (wstat 256, 0x100)
    Failed 1/26 subtests

To avoid surprises in the semantics of the DSL, the scenarios should be
run in the order they were defined in the .t file. Doing this might
let us simplify the implementation later.

Refactor the way subtsets are selected or disabled using environment
variables. We also enforce the existence of scenario names listed in
those environment variables, which we previously didn’t.

Re-engineer the AST data structure generated by TestUtil::DSL so that
multiple root_hints are allowed. Subsequent root_hints entirely override
previous ones.

In order to avoid surprises for the end user, it’s better to respect the
order in which the expect and forbid keywords were defined in the
scenarios too. It simplifies the DSL’s implementation as well.

Add a keyword to tell Zonemaster::Engine::Resolver to clear the cache
between tests. That can be useful in some corner cases where caching of
resolution failures interferes with the correct operation of the unit
test, which is the case of t/Test-connectivity04.t, among others.

This one is interesting because it has a scenario, PTR-IS-ILLEGAL-CNAME,
which expects one tag, forbids one tag and leaves two others hanging.

Rewriting this test file to use the DSL helped me find a typo in the
original t/Test-basic02.t file that caused the GOOD-2 scenario to point
to good-1.basic02.xa instead of good-2.basic02.xa (see PR zonemaster#1465).

It’s a good demonstration of why it can be important to design data
structures and formats in such a way that internal repetition is avoided
unless absolutely necessary: when done right, it makes subtle errors
like these more obvious.

In this specific case, the unit test was broken in two files because
caching of resolution failures (“blacklisting”) caused issues with one
of the scenarios.

Thanks to the DSL’s “clear_cache” keyword, however, we can put the
affected scenario in the same file.

This one is interesting because it has a few scenarios that are marked
as not testable. We do the same here.

Another rewrite of a unit test that led me to uncover a typo in the
original unit test. Actually, this time, the specification was
incorrect (see zonemaster/zonemaster#1414).

This one is fairly straightforward, but many scenarios are marked as not
testable.