Skip to content

Add support for crashtests#20017

Merged
jgraham merged 8 commits intomasterfrom
crashtests
Dec 12, 2019
Merged

Add support for crashtests#20017
jgraham merged 8 commits intomasterfrom
crashtests

Conversation

@jgraham
Copy link
Contributor

@jgraham jgraham commented Oct 31, 2019

Copy link
Contributor

@marcoscaceres marcoscaceres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving payment request test rename. Thanks!

Copy link
Contributor

@FremyCompany FremyCompany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first glance, the css-tables related renames seem ok

Copy link
Contributor

@svgeesus svgeesus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The renames to avoid -crash where it shouldn't be seem good. I skimmed over the python changes but someone else should review those.

Copy link
Member

@gsnedders gsnedders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Files that shouldn't have been renamed:

  • adjoining-float-new-fc-thcrash.html (doesn't meet the criteria in the commit message)
  • gradient-thcrash.html (is a reftest?!)
  • extremely-tall-multicol-with-extremely-tall-child-thcrash.html (doesn't meet the criteria in the commit message)
  • before-in-display-none-thcrash.html (ditto)
  • shape-outside-inset-thcrash.html (is a reftest)

I'm not gonna bother going through the rest, something has gone wrong…

Copy link
Member

@gsnedders gsnedders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

css/css-position/position-absolute-crash-chrome-005.html and css/css-scoping/whitespace-crash-001.html don't need renamed (they're not crash tests).

I also wonder if we should have a lint for -crash tests containing testharness.js? It really is an antipattern, but also something Chrome developers have already done (and AFAIK they run differently as a result in run-webkit-tests, @Hexcles?).

@jgraham
Copy link
Contributor Author

jgraham commented Dec 11, 2019

css/css-position/position-absolute-crash-chrome-005.html and css/css-scoping/whitespace-crash-001.html don't need renamed (they're not crash tests).

Fixed.

@Hexcles
Copy link
Member

Hexcles commented Dec 11, 2019

css/css-position/position-absolute-crash-chrome-005.html and css/css-scoping/whitespace-crash-001.html don't need renamed (they're not crash tests).

They could be rewritten to real tests though, but that's for another day.

I also wonder if we should have a lint for -crash tests containing testharness.js?

I agree. We should.

It really is an antipattern, but also something Chrome developers have already done (and AFAIK they run differently as a result in run-webkit-tests, @Hexcles?).

Sorry I'm not sure what you're asking? Until today's introduction of crash tests, Chrome developers have written some de facto crash tests with dummy testharness asserts in them. These should not be needed in the future (at least not named with -crash). However, it is true that run_web_tests might need to be updated to support these assert-free crash tests (well, more importantly the lack of testharnessreport.js, due to a hack we use internally).

@jgraham
Copy link
Contributor Author

jgraham commented Dec 12, 2019

I also wonder if we should have a lint for -crash tests containing testharness.js?

I agree. We should.

I think we could leave that to a followup? It doesn't seem critical to landing the patch, and there are already a bunch of exceptions which we could/should fix at the same time, but deciding how to proceed with that is more work.

@gsnedders
Copy link
Member

Until today's introduction of crash tests, Chrome developers have written some de facto crash tests with dummy testharness asserts in them.

AIUI, does run-webkit-tests not run -crash tests in WPT as actual crash tests already? i.e., it ignores any asserts (or failing asserts) in them. My concern is people adding new tests like that with actual asserts.

But to me I'm happy to leave it as a followup.

@Hexcles
Copy link
Member

Hexcles commented Dec 12, 2019

AIUI, does run-webkit-tests not run -crash tests in WPT as actual crash tests already? i.e., it ignores any asserts (or failing asserts) in them. My concern is people adding new tests like that with actual asserts.

That's new to me, but it's possible. And yes we should prevent people from adding dummy asserts to crash tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants