testsuite: disable test_dropped_events on Windows#13476
Conversation
dra27
left a comment
There was a problem hiding this comment.
This test has not been failing regular on mingw-w64 - if it's disabled at all, it should be for MSVC only.
|
I don't see an easy way to disable a test on MSVC only (ocamltest has a |
|
No - that means we add the |
|
Note that this test has been failing regularly on ppc-little endian port on the Inria CI too. |
|
@dra27 I reviewed your code and believe it is correct -- I approve, but the Github UI wouldn't let me as the original submitter. Would you approve on my behalf, and merge? |
dra27
left a comment
There was a problem hiding this comment.
OK! I haven't done a proper analysis of the logs coming from Inria, but some simple searching on the failure logs suggests that this has started failing possibly since the beginning of August?
What's concerning me is that we were not seeing this test failing lots when originally restoring MSVC (merged in March) or with clang-cl (merged in April). The test though does combine some interesting facets - it's a lot of callback heavy stuff in a multi-domain context with intentionally competing loops.
What's slightly concerning is that June-August is around the time when the callback machinery was being overhauled, etc. and whether it's simply runtime-events which happens to be revealing a problem in that?
|
Yes, I agree that there is something worth investigating here, the point is not to kill the messenger, but I would like to silence it so that we are not inconvenienced in all the other work we are doing. |
lib-runtime-events/test_dropped_events is flaky on Windows in trunk, and it makes dealing with new PRs annoying (for example it makes the CI fail for clang-cl on #13467 and #13475). There is an investigation of this test in #12397, but identifying and fixing the issue may take a while, and in the meantime we have a broken CI in trunk and this is bad.
The present PR disables the test on Windows, to hide the failure away and resume normal development life where CI is green. The failure is probably due to a real bug that should be fixed, but this bug should be only a problem for people who maintain runtime-events, not for everyone.
(Maybe the test is more fragile than just Windows and should be disabled entirely, I propose to wait to see if it is flaky in practice on non-Windows system to move to that.)