Create Basic04#859
Conversation
|
@vlevigneron, any comment on this PR before I merge? |
There was a problem hiding this comment.
@matsduf I just want to be sure we have the same understanding of this change. The description of this new test is reasonnable for me and I have, AFAICR, no issue to implement it as is. But, if we want to execute it ALL the time before another test, we have to change many things in the code. In fact, Basic tests are only run when we do a complete Check with all the tests. It is not used when we just want to execute all tests of a module or a specific test. I'm not against those changes, but it implies other specs modifications, I guess.
|
@vlevigneron, good point. I had the impression that Basic was always run before all other tests, but as I understand it now, the special thing about Basic is that it is run first and that the test will break if some of the Basic test cases fail. I do not think we should change that, especially not as of this PR. The main use case is when all test cases are run. If we should change anything it should probably be in the way CLI runs selected test cases or modules, but that would be a different issue/PR. I will do a slight update to the wording of this PR (test case) so that you do not get the impression that this test case is automatically run before other modules. I will also look at my other PR where I refer to Basic04. |
|
@matsduf No, as I told you, Basic is only called in ONE case, when we run all the tests (method Zonemaster::Engine::test_zone()) which is the use case you are talking about. If we plan to run it in this only case, I'll hav to find a way to prevent that same tests are executed twice (first in Basic04, then in test case methods). |
@vlevigneron, my idea is that we should remove those tests (or rather messages) from the other test cases. Most test cases that I have updated (available PRs) have the equivalent tests (messages) removed. I do not think that we should invent something magic. Basic04 should, in the future, be the only test case that complain about unavailability. If you look at the other PRs you can see that I have tried to avoid to complain about the same thing several times. |
|
@vlevigneron, can you approve this? |
This test case will test some basic nameserver and zone functionality. Since it is in "basic" it will always run.
Many test cases will query the name servers in the delegation or the
name servers appointed by the NS records in the zone for the NS or SOA
record or both. Reporting problem is crucial, but instead of letting
several test cases report the same problems found, all test cases are
made dependent on this test case and this case only will report any
problems found in the following areas:
section in the response on a SOA query for Child Zone.
section in the response on an NS query for Child Zone.
answer section.
on query for SOA or NS for Child Zone.
This test case is assumed to be run before Basic02 to be able to capture the odd, but existing, situation where all name servers listen to TCP, but not UDP.
When this test case has been created, other test cases could be updated so that any message reporting the same error could be suppressed or having its level lowered to DEBUG or DEBUG2.
All message tags in this test case start with "B04_" to firstly make them test case local, and secondly make it easier for the user to find where it comes from. There is no intention that any Zonemaster code should do anything "magical" with that prefix.