Enable css-grid WPT tests by nicoburns · Pull Request #34052 · servo/servo

nicoburns · 2024-10-29T10:05:18Z

Changes made:

Enable css/css-grid WPT tests for layout2020
Update test assertions for these tests

Does layout2013 also need these to be enabled?

./mach build -d does not report any errors
./mach test-tidy does not report any errors
There are tests for these changes

Loirooriol · 2024-10-29T10:29:45Z

Does layout2013 also need these to be enabled?

I think this is already enabling for both 2020 and 2013. We don't really check 2013 results other than when importing new tests, but if you can include them here then better.

mrobinson · 2024-10-29T10:29:57Z

It isn't necessary to generate results for Layout 2013, as they will be updated when the weekly WPT job happens.

github-actions · 2024-10-29T10:29:58Z

🔨 Triggering try run (#11571976416) for Linux WPT

github-actions · 2024-10-29T10:30:13Z

🔨 Triggering try run (#11571981708) for Linux WPT

Loirooriol · 2024-10-29T10:31:03Z

Dang, conflict with the tags

github-actions · 2024-10-29T11:11:57Z

⚠️ Try run (#11571981708) failed.

github-actions · 2024-10-29T11:16:20Z

🔨 Triggering try run (#11572728879) for Linux WPT

github-actions · 2024-10-29T11:43:36Z

⚠️ Try run (#11572728879) failed.

github-actions · 2024-10-29T13:12:18Z

🔨 Triggering try run (#11574585902) for Linux WPT

github-actions · 2024-10-29T13:38:16Z

Test results for linux-wpt-layout-2020 from try job (#11574585902):

Flaky unexpected result (12)

TIMEOUT [expected OK] /_webgl/conformance/glsl/misc/shader-uniform-packing-restrictions.html (#28103)
TIMEOUT [expected OK] /_webgl/conformance/glsl/misc/shader-with-non-reserved-words.html (#16216)
- NOTRUN [expected PASS] subtest: Overall test
OK /css/cssom-view/MediaQueryList-addListener-removeListener.html (#24569)
- FAIL [expected PASS] subtest: listeners are called in order they were added
```
assert_array_equals: lengths differ, expected array ["1st", "2nd"] length 2, got [] length 0
```
OK /custom-elements/form-associated/ElementInternals-setFormValue.html (#29174)
- PASS [expected FAIL] subtest: Single value - name is missing
CRASH [expected OK] /encoding/legacy-mb-tchinese/big5/big5-decode-big5-hkscs.html?1-1000
TIMEOUT [expected OK] /fetch/api/redirect/redirect-keepalive.https.any.html (#32153)
- TIMEOUT [expected PASS] subtest: [keepalive][iframe][load] mixed content redirect; setting up
```
Test timed out
```
OK /html/browsers/browsing-the-web/navigating-across-documents/initial-empty-document/iframe-src-aboutblank-navigate-immediately.html (#29048)
- PASS [expected FAIL] subtest: Navigating to a different document with link click
OK /html/browsers/windows/embedded-opener-remove-frame.html (#23867)
- PASS [expected FAIL] subtest: opener of discarded auxiliary browsing context
OK /html/syntax/parsing/DOMContentLoaded-defer.html (#21550)
- PASS [expected FAIL] subtest: The end: DOMContentLoaded and defer scripts
OK /workers/WorkerGlobalScope-close.html (#23064)
- PASS [expected FAIL] subtest: Test sending a message after closing.
OK [expected ERROR] /workers/constructors/Worker/Worker-constructor.html (#22991)
OK /xhr/open-url-multi-window-5.htm (#23360)
- FAIL [expected PASS] subtest: XMLHttpRequest: open() resolving URLs (multi-Window; 5)
```
assert_throws_dom: function "function() {client.open("GET", "...") }" did not throw
```

Stable unexpected results that are known to be intermittent (14)

FAIL [expected PASS] /_mozilla/css/iframe/hide_and_show.html (#15265)
PASS [expected FAIL] /css/css-values/vh_not_refreshing_on_chrome.html (#23385, #15570)
OK /fetch/metadata/generated/css-font-face.https.sub.tentative.html (#32732)
- FAIL [expected PASS] subtest: sec-fetch-dest
```
promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."
```
OK /html/browsers/browsing-the-web/navigating-across-documents/navigation-unload-cross-origin.sub.window.html (#29056)
- FAIL [expected PASS] subtest: Cross-origin navigation started from unload handler must be ignored
```
promise_test: Unhandled rejection with value: object "SecurityError: The operation is insecure."
```
OK /html/semantics/embedded-content/the-iframe-element/iframe-loading-lazy-nav-location-replace-set-src.html (#32697)
- PASS [expected FAIL] subtest: Navigating iframe loading='lazy' and then setting src: location.replace
TIMEOUT [expected CRASH] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping-3.html (#24057)
CRASH [expected TIMEOUT] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-1.html (#24066)
CRASH [expected TIMEOUT] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-2.html (#22154)
TIMEOUT [expected OK] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-3.html (#24066)
OK /html/webappapis/dynamic-markup-insertion/document-write/module-static-import-delayed.html (#26243)
- PASS [expected FAIL] subtest: document.write in an imported module
OK /navigation-timing/test-navigation-type-reload.html (#33334)
- FAIL [expected PASS] subtest: Reload domComplete > Original domComplete
```
assert_true: Reload domComplete &gt; Original domComplete expected true got false
```
- FAIL [expected PASS] subtest: Reload domContentLoadedEventEnd > Original domContentLoadedEventEnd
```
assert_true: Reload domContentLoadedEventEnd &gt; Original domContentLoadedEventEnd expected true got false
```
- FAIL [expected PASS] subtest: Reload loadEventEnd > Original loadEventEnd
```
assert_true: Reload loadEventEnd &gt; Original loadEventEnd expected true got false
```
- FAIL [expected PASS] subtest: Reload loadEventStart > Original loadEventStart
```
assert_true: Reload loadEventStart &gt; Original loadEventStart expected true got false
```
OK [expected TIMEOUT] /webmessaging/with-ports/017.html (#24486)
- PASS [expected TIMEOUT] subtest: origin of the script that invoked the method, about:blank
OK [expected TIMEOUT] /webmessaging/without-ports/018.html (#24485)
- PASS [expected TIMEOUT] subtest: origin of the script that invoked the method, javascript:
TIMEOUT [expected OK] /webstorage/localstorage-about-blank-3P-iframe-opens-3P-window.partitioned.html (#29053)
- TIMEOUT [expected PASS] subtest: StorageKey: test 3P about:blank window opened from a 3P iframe
```
Test timed out
```

Stable unexpected results (1)

OK [expected TIMEOUT] /css/css-grid/layout-algorithm/grid-flex-track-intrinsic-sizes-003.html
- FAIL [expected PASS] subtest: auto min item 'grid' with: grid-template-columns: 0fr; and grid-template-rows: 0fr;
```
assert_in_array: gridTemplateColumns value undefined not in array ["100px"]
```
- FAIL [expected PASS] subtest: auto min item 'grid' with: grid-template-columns: 1fr; and grid-template-rows: 1fr;
```
assert_in_array: gridTemplateColumns value undefined not in array ["100px"]
```
- FAIL [expected PASS] subtest: auto min item 'grid' with: grid-template-columns: 2fr; and grid-template-rows: 2fr;
```
assert_in_array: gridTemplateColumns value undefined not in array ["100px"]
```
- FAIL [expected PASS] subtest: auto min item 'grid' with: grid-template-columns: minmax(0, 0fr); and grid-template-rows: minmax(0, 0fr);
```
assert_in_array: gridTemplateColumns value undefined not in array ["0px"]
```
- FAIL [expected PASS] subtest: auto min item 'grid' with: grid-template-columns: minmax(0, .5fr); and grid-template-rows: minmax(0, .5fr);
```
assert_in_array: gridTemplateColumns value undefined not in array ["30px"]
```
- FAIL [expected PASS] subtest: auto min item 'grid' with: grid-template-columns: minmax(0, 1fr); and grid-template-rows: minmax(0, 1fr);
```
assert_in_array: gridTemplateColumns value undefined not in array ["60px"]
```
- FAIL [expected PASS] subtest: auto min item 'grid' with: grid-template-columns: minmax(0, 2fr); and grid-template-rows: minmax(0, 2fr);
```
assert_in_array: gridTemplateColumns value undefined not in array ["60px"]
```
- FAIL [expected PASS] subtest: auto min item 'grid' with: grid-template-columns: minmax(75px, 1fr); and grid-template-rows: minmax(75px, 1fr);
```
assert_in_array: gridTemplateColumns value undefined not in array ["75px"]
```
- FAIL [expected PASS] subtest: auto min item 'grid' with: grid-template-columns: 0fr 0fr; and grid-template-rows: 0fr 0fr;
```
assert_in_array: gridTemplateColumns value undefined not in array ["0px 0px"]
```
- FAIL [expected PASS] subtest: auto min item 'grid' with: grid-template-columns: 0fr 1fr; and grid-template-rows: 0fr 1fr;
```
assert_in_array: gridTemplateColumns value undefined not in array ["0px 60px"]
```
- And 92 more unexpected results...

github-actions · 2024-10-29T13:38:30Z

⚠️ Try run (#11574585902) failed.

nicoburns · 2024-10-29T14:01:59Z

The failing test seems to be unreliable. Sometimes it times out, sometimes it completes. What's the procedure for such tests?

mrobinson · 2024-10-29T14:06:16Z

The failing test seems to be unreliable. Sometimes it times out, sometimes it completes. What's the procedure for such tests?

Usually when this is the case the test is marked as having both possible results with a WPT .ini file. On the other hand, without incremental layout we typically don't expect this kind of non-deterministic behavior in layout. Do you have a sense for what might causing the flaky behavior?

Loirooriol · 2024-10-29T14:06:43Z

I think expected: [TIMEOUT, OK] in the .ini does the trick, or you could file an issue with I-intermittent.

mrobinson · 2024-10-29T14:06:52Z

Usually when this is the case the test is marked as having both possible results with a WPT .ini file. On the other hand, without incremental layout we typically don't expect this kind of non-deterministic behavior in layout. Do you have a sense for what might causing the flaky behavior?

Oh, a case this might happen in layout is if the test case is very heavy and the algorithm used for layout is very slow.

nicoburns · 2024-10-29T14:32:30Z

Oh, a case this might happen in layout is if the test case is very heavy and the algorithm used for layout is very slow.

Hmm... this is a CSS Grid test, but of course Servo doesn't have grid support yet, so I'd expect this to be falling back to block layout, which shouldn't be slow? This test does have quite a few test cases (script driven) though.

github-actions · 2024-10-29T14:33:06Z

🔨 Triggering try run (#11576102714) for Linux WPT

github-actions · 2024-10-29T15:01:14Z

Test results for linux-wpt-layout-2020 from try job (#11576102714):

Flaky unexpected result (20)

FAIL [expected PASS] /_mozilla/mozilla/iframe/resize_after_load.html (#13573)
TIMEOUT [expected OK] /_webgl/conformance/glsl/misc/shader-uniform-packing-restrictions.html (#28103)
CRASH [expected OK] /encoding/legacy-mb-korean/euc-kr/euckr-decode-ks_c_5601-1987.html?11001-12000
OK /html/browsers/browsing-the-web/navigating-across-documents/javascript-url-return-value-handling-dynamic.html (#28066)
- FAIL [expected PASS] subtest: 0041 set in href="" targeting a frame and clicked
```
assert_equals: expected "A" but got ""
```
- FAIL [expected PASS] subtest: 0080 00FF set in href="" targeting a frame and clicked
```
assert_equals: expected "�ÿ" but got ""
```
OK /html/browsers/browsing-the-web/navigating-across-documents/navigation-unload-same-origin.window.html (#29049)
- FAIL [expected PASS] subtest: Same-origin navigation started from unload handler must be ignored
```
assert_equals: expected "?pass" but got "?fail"
```
TIMEOUT [expected OK] /html/browsers/history/the-history-interface/traverse-during-unload.html (#28688)
- TIMEOUT [expected PASS] subtest: Traversing the history during unload
```
Test timed out
```
OK /html/browsers/history/the-history-interface/traverse_the_history_3.html (#21383)
- PASS [expected FAIL] subtest: Multiple history traversals, last would be aborted
OK /html/browsers/history/the-history-interface/traverse_the_history_5.html (#21383)
- PASS [expected FAIL] subtest: Multiple history traversals, last would be aborted
OK /html/browsers/the-window-object/open-close/creating_browsing_context_test_01.html (#29046)
- PASS [expected FAIL] subtest: first argument: absolute url
TIMEOUT [expected OK] /html/interaction/focus/the-autofocus-attribute/document-with-fragment-nonexistent.html (#28259)
- TIMEOUT [expected FAIL] subtest: Autofocus elements in top-level browsing context's documents with non-existent fragments should work.
```
Test timed out
```
TIMEOUT [expected OK] /html/semantics/document-metadata/the-base-element/base_target_does_not_affect_location_assignment.html (#32615)
- TIMEOUT [expected PASS] subtest: base_target_does_not_affect_location_assignment
```
Test timed out
```
OK /html/semantics/embedded-content/the-iframe-element/iframe-loading-lazy-nav-location-replace-set-src.html (#32697)
- PASS [expected FAIL] subtest: Navigating iframe loading='lazy' and then setting src: location.replace
OK /html/semantics/embedded-content/the-iframe-element/iframe-loading-lazy-reload-location-reload.html (#32595)
- PASS [expected FAIL] subtest: Reloading iframe loading='lazy' before it is loaded: location.reload
OK /html/semantics/forms/historical.html (#28568)
- PASS [expected FAIL] subtest: <input name=isindex> should not be supported
CRASH [expected OK] /html/semantics/forms/the-fieldset-element/disabled-003.html (#31730)
OK /html/syntax/parsing/DOMContentLoaded-defer.html (#21550)
- PASS [expected FAIL] subtest: The end: DOMContentLoaded and defer scripts
TIMEOUT [expected OK] /resource-timing/nested-context-navigations-iframe.html (#24311)
- TIMEOUT [expected PASS] subtest: Test that iframe navigations are not observable by the parent, even after history navigations by the parent
```
Test timed out
```
- NOTRUN [expected PASS] subtest: Test that crossorigin iframe navigations are not observable by the parent, even after history navigations by the parent
- NOTRUN [expected PASS] subtest: Test that cross-site iframe navigations are not observable by the parent, even after history navigations by the parent
- NOTRUN [expected PASS] subtest: Test that iframe navigations are not observable by the parent
- NOTRUN [expected PASS] subtest: Test that crossorigin iframe navigations are not observable by the parent
- NOTRUN [expected PASS] subtest: Test that cross-site iframe navigations are not observable by the parent
- NOTRUN [expected PASS] subtest: Test that iframe refreshes are not observable by the parent
- NOTRUN [expected PASS] subtest: Test that crossorigin iframe refreshes are not observable by the parent
- NOTRUN [expected PASS] subtest: Test that cross-site iframe refreshes are not observable by the parent
CRASH [expected TIMEOUT] /resource-timing/response-status-code.html (#29309)
TIMEOUT [expected OK] /webmessaging/with-ports/018.html (#24485)
- TIMEOUT [expected PASS] subtest: origin of the script that invoked the method, javascript:
```
Test timed out
```
TIMEOUT [expected OK] /webstorage/localstorage-about-blank-3P-iframe-opens-3P-window.partitioned.html (#29053)
- TIMEOUT [expected PASS] subtest: StorageKey: test 3P about:blank window opened from a 3P iframe
```
Test timed out
```

Stable unexpected results that are known to be intermittent (16)

FAIL [expected PASS] /_mozilla/css/iframe/hide_and_show.html (#15265)
OK /css/cssom-view/MediaQueryList-addListener-removeListener.html (#24569)
- FAIL [expected PASS] subtest: listeners are called correct number of times
```
assert_equals: expected 1 but got 0
```
TIMEOUT [expected OK] /fetch/api/redirect/redirect-keepalive.https.any.html (#32153)
- TIMEOUT [expected PASS] subtest: [keepalive][iframe][load] mixed content redirect; setting up
```
Test timed out
```
OK /fetch/metadata/generated/css-font-face.https.sub.tentative.html (#32732)
- FAIL [expected PASS] subtest: sec-fetch-dest
```
promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."
```
OK /html/browsers/browsing-the-web/navigating-across-documents/navigation-unload-cross-origin.sub.window.html (#29056)
- FAIL [expected PASS] subtest: Cross-origin navigation started from unload handler must be ignored
```
promise_test: Unhandled rejection with value: object "SecurityError: The operation is insecure."
```
TIMEOUT [expected CRASH] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping-1.html (#22647)
- TIMEOUT [expected FAIL] subtest: Check that popups from a sandboxed iframe escape the sandbox if allow-popups-to-escape-sandbox is used
```
Test timed out
```
TIMEOUT [expected CRASH] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping-2.html (#22667)
TIMEOUT [expected CRASH] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping-3.html (#24057)
CRASH [expected TIMEOUT] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-1.html (#24066)
OK [expected TIMEOUT] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-2.html (#22154)
- FAIL [expected NOTRUN] subtest: Check that popups from a sandboxed iframe do not escape the sandbox
```
assert_equals: It came from a sandboxed iframe expected "null" but got "http://web-platform.test:8000"
```
OK /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-3.html (#24066)
- FAIL [expected NOTRUN] subtest: Check that popups from a sandboxed iframe do not escape the sandbox
```
assert_equals: It came from a sandboxed iframe expected "null" but got "http://web-platform.test:8000"
```
TIMEOUT [expected OK] /html/semantics/forms/form-submission-0/reparent-form-during-planned-navigation-task.html (#29724)
- TIMEOUT [expected PASS] subtest: reparent-form-during-planned-navigation-task
```
Test timed out
```

OK /html/webappapis/update-rendering/child-document-raf-order.html (#33028)

FAIL [expected PASS] subtest: Ordering of steps in "Update the Rendering" - child document requestAnimationFrame order

assert_array_equals: expected order of notifications lengths differ, expected array ["parent_raf", "first_child_raf", "second_child_raf"] length 3, got ["parent_raf", "first_child_raf", "second_child_raf", "parent_raf", "first_child_raf", "second_child_raf"] length 6

OK /navigation-timing/test-navigation-type-reload.html (#33334)
- FAIL [expected PASS] subtest: Reload domComplete > Original domComplete
```
assert_true: Reload domComplete &gt; Original domComplete expected true got false
```
- FAIL [expected PASS] subtest: Reload domContentLoadedEventEnd > Original domContentLoadedEventEnd
```
assert_true: Reload domContentLoadedEventEnd &gt; Original domContentLoadedEventEnd expected true got false
```
- FAIL [expected PASS] subtest: Reload loadEventEnd > Original loadEventEnd
```
assert_true: Reload loadEventEnd &gt; Original loadEventEnd expected true got false
```
- FAIL [expected PASS] subtest: Reload loadEventStart > Original loadEventStart
```
assert_true: Reload loadEventStart &gt; Original loadEventStart expected true got false
```
OK [expected TIMEOUT] /webmessaging/with-ports/017.html (#24486)
- PASS [expected TIMEOUT] subtest: origin of the script that invoked the method, about:blank
OK [expected TIMEOUT] /webmessaging/without-ports/018.html (#24485)
- PASS [expected TIMEOUT] subtest: origin of the script that invoked the method, javascript:

github-actions · 2024-10-29T15:01:26Z

✨ Try run (#11576102714) succeeded.

nicoburns · 2024-10-29T15:28:13Z

I think expected: [TIMEOUT, OK] in the .ini does the trick

Thanks! That does seems to have done the trick :)

Signed-off-by: Nico Burns <nico@nicoburns.com>

nicoburns · 2024-10-30T15:49:11Z

@mrobinson Is anything still blocking this one? CI / WPT is passing now.

nicoburns added the A-testing label Oct 29, 2024

nicoburns requested a review from mrobinson October 29, 2024 10:05

nicoburns force-pushed the css-grid-tests branch from b3a6bee to 6de4563 Compare October 29, 2024 10:05

mrobinson added the T-linux-wpt Do a try run of the WPT label Oct 29, 2024

Loirooriol added T-linux-wpt-2013 and removed T-linux-wpt Do a try run of the WPT labels Oct 29, 2024

github-actions bot removed the T-linux-wpt-2013 label Oct 29, 2024

Loirooriol added T-linux-wpt-2013 T-linux-wpt Do a try run of the WPT labels Oct 29, 2024

github-actions bot removed T-linux-wpt-2013 T-linux-wpt Do a try run of the WPT labels Oct 29, 2024

nicoburns added T-linux-wpt-2013 T-linux-wpt Do a try run of the WPT labels Oct 29, 2024

github-actions bot removed T-linux-wpt-2013 T-linux-wpt Do a try run of the WPT labels Oct 29, 2024

nicoburns added the T-linux-wpt Do a try run of the WPT label Oct 29, 2024

github-actions bot removed the T-linux-wpt Do a try run of the WPT label Oct 29, 2024

nicoburns force-pushed the css-grid-tests branch from 6c0b233 to 77a4361 Compare October 29, 2024 14:00

nicoburns force-pushed the css-grid-tests branch from 265bea9 to d6169c1 Compare October 29, 2024 14:31

nicoburns added the T-linux-wpt Do a try run of the WPT label Oct 29, 2024

github-actions bot removed the T-linux-wpt Do a try run of the WPT label Oct 29, 2024

nicoburns added 4 commits October 30, 2024 12:02

Enable css-grid WPT tests

e70e6a4

Signed-off-by: Nico Burns <nico@nicoburns.com>

Add assertions for css-grid WPT tests

acaeea5

Signed-off-by: Nico Burns <nico@nicoburns.com>

Add test assertions

116e903

Signed-off-by: Nico Burns <nico@nicoburns.com>

Allow for timeouts

8546220

Signed-off-by: Nico Burns <nico@nicoburns.com>

nicoburns force-pushed the css-grid-tests branch from d6169c1 to 8546220 Compare October 30, 2024 05:03

mrobinson approved these changes Oct 30, 2024

View reviewed changes

mrobinson added this pull request to the merge queue Oct 30, 2024

Merged via the queue into servo:main with commit 635c10a Oct 30, 2024

nicoburns mentioned this pull request Dec 4, 2024

Tracking issue: CSS Grid #34479

Open

20 tasks

Uh oh!

Conversation

nicoburns commented Oct 29, 2024

Uh oh!

Loirooriol commented Oct 29, 2024

Uh oh!

mrobinson commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024

Uh oh!

Loirooriol commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024

Uh oh!

nicoburns commented Oct 29, 2024

Uh oh!

mrobinson commented Oct 29, 2024

Uh oh!

Loirooriol commented Oct 29, 2024

Uh oh!

mrobinson commented Oct 29, 2024

Uh oh!

nicoburns commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024

Uh oh!

nicoburns commented Oct 29, 2024

Uh oh!

nicoburns commented Oct 30, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants