Currently three incremental test cases fail when the cfg comparison is activated. The first two throw a Not_found exception, for the third the result is not as expected.
Since the cfg comparison often breaks when new features are introduced and the only the AST comparison is considered when adapting the incremental pre-processing for the solver, I would suggest to run the incremental tests also with the cfg comparison in the regular regression test runs.
Testing 03/02 precision-annotation/reluctant-int-annotation Status: 2 (exception)
- Timings:
- TOTAL 0.007 s
- parse 0.002 s
- convert to CIL 0.003 s
- mergeCIL 0.002 s
- compareCilFiles 0.001 s
- createCFG 0.001 s
- handle 0.001 s
- iter_connect 0.001 s
- computeSCCs 0.001 s
Testing 03/03 precision-annotation/reluctant-int-annotation-dyn Status: 2 (exception)
- Timings:
- TOTAL 0.008 s
- parse 0.003 s
- convert to CIL 0.003 s
- mergeCIL 0.002 s
- compareCilFiles 0.001 s
- createCFG 0.001 s
- handle 0.001 s
- iter_connect 0.001 s
- computeSCCs 0.001 s
03/02 failed!
Expected assert, but registered nothing on reluctant-int-annotation-dyn:10
__goblint_check(in == 17);
Expected fail, but registered nothing on reluctant-int-annotation-dyn:21
__goblint_check(a); // FAIL!
Expected assert, but registered nothing on reluctant-int-annotation-dyn:23
__goblint_check(a == 17);
03/03 failed!
Expected norace, but registered race on race-call-remove:7
g++; // NORACE (unique thread)
Expected norace, but registered race on race-call-remove:12
g++; // NORACE
13/05 failed!
3 test(s) failed: ["03/02 reluctant-int-annotation", "03/03 reluctant-int-annotation-dyn", "13/05 race-call-remove"]
Currently three incremental test cases fail when the cfg comparison is activated. The first two throw a
Not_foundexception, for the third the result is not as expected.Since the cfg comparison often breaks when new features are introduced and the only the AST comparison is considered when adapting the incremental pre-processing for the solver, I would suggest to run the incremental tests also with the cfg comparison in the regular regression test runs.