Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify tests with pytest-playwright #523

Merged
merged 9 commits into from Jun 21, 2022
Merged

Simplify tests with pytest-playwright #523

merged 9 commits into from Jun 21, 2022

Conversation

philippjfr
Copy link
Contributor

The pytest-playwright package simplifies tests by providing a page fixture that launches the browser for you and also easily lets you configure playwright in the pytest command e.g. to switch to headed (rather than headless) mode and slow the test down and see what it's actually doing.

@philippjfr philippjfr added the tag: tests Related to tests label Jun 17, 2022
Copy link
Collaborator

@fpliger fpliger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice! thank you!

@fpliger
Copy link
Collaborator

fpliger commented Jun 17, 2022

weird.. not sure why CI can'd find the package to install. Aren't we rebuilding the env every run?

@philippjfr
Copy link
Contributor Author

Ah, looks like it's not actually on conda-forge yet. May push up a recipe on of these days.

@maximlt
Copy link
Contributor

maximlt commented Jun 17, 2022

I opened an issue to get a conda package published some time ago, no reaction yet: microsoft/playwright-pytest#113

It isn't going to be that straightforward as the pytest plugin depends on Playwright which itself isn't release on conda-forge but on the Microsoft channel. I assume first Playwright needs to have a recipe on conda-forge?

@marimeireles
Copy link
Member

Thanks for opening the issue @maximlt. Do you know if that's the only way of getting packages merged to their channel?

@maximlt
Copy link
Contributor

maximlt commented Jun 20, 2022

No :/
I guess for now I'd just change the environment.yml so that pytest-playwright is installed with pip.

@marimeireles
Copy link
Member

Seems like an acceptable work around for us to merge this, what's your opinion @philippjfr?

@philippjfr
Copy link
Contributor Author

Agree with that, it's also what we do in the Panel test suite.

@philippjfr
Copy link
Contributor Author

Hmm, the clock example either seems flakey or broken, it's failing for me on #522 as well.

@marimeireles
Copy link
Member

Ah okay, but the problem with pytest-playwright is not replicable anymore right?
I'll rollback my last commit, cause I thought it was weird the previous one didn't work 😅

@philippjfr
Copy link
Contributor Author

Yeah, I think you can roll that one back.

@marimeireles
Copy link
Member

Since Fabio approved this, I'll merge :) once the CI says it's okay again (just out of superstition 🔮 👀 )

@philippjfr
Copy link
Contributor Author

So confused about pre-commit, locally I see no issues.

@philippjfr
Copy link
Contributor Author

philippjfr commented Jun 21, 2022

Okay, I think what happened here is that some tests actually run quite quickly and the playwright call that checks for the loading screen runs after it has already been executed.

@marimeireles
Copy link
Member

@philippjfr I don't think this is a CI only problem, because an user is seeing this locally too. #528

@philippjfr
Copy link
Contributor Author

Yeah, the tests as written are just quite flakey. Trying to improve that but it's basically a race condition.

@marimeireles
Copy link
Member

I can reproduce this every time locally, independently of the tests. I'm just running the example.
It happens with alpha and with the local build, on FF and chrome.

@philippjfr
Copy link
Contributor Author

I can reproduce this every time locally, independently of the tests. I'm just running the example.

To be clear what is "this" you are referring to? I think we're talking about different things.

@philippjfr
Copy link
Contributor Author

The type annotations are indeed still an issue but we don't have any tests checking them anymore.

@marimeireles
Copy link
Member

marimeireles commented Jun 21, 2022

Ah, sorry, @philippjfr.
I'm talking about this issue: #528. I thought it was the culprit for the CI crash.
What are you talking about?

@marimeireles
Copy link
Member

marimeireles commented Jun 21, 2022

I understand that you're making the tests sturdier, but I thought we were also fixing this issue 😅 sorry.

@philippjfr
Copy link
Contributor Author

@marimeireles
Copy link
Member

So we are talking about the same thing :)
I'm also trying to understand this.

@philippjfr
Copy link
Contributor Author

I'm now hoping it was down to the custom (and pointless) <py-config> that was in the simple_clock.html.

@philippjfr
Copy link
Contributor Author

Tests passing 🥳 Maybe let's restart the test one more time to make sure it wasn't just a fluke.

@marimeireles
Copy link
Member

marimeireles commented Jun 21, 2022

Hum... I'm not sure, honestly.
Strangely enough, the example wasn't working for me, but now it is, just like now it is for the CI. (EDIT: And I was running the local build version)
But I still have the <py-config> tag...
I'm not sure if we fixed it, but I can't reproduce it anymore.

@philippjfr
Copy link
Contributor Author

Weird, always worked for me locally. Just very confused by the whole thing.

@marimeireles
Copy link
Member

Well, I'll re-run the tests and if it's good we merge this.
I don't think it's related to the contents of this PR anyway but rather an issue with the example.
Thank you for your effort here Philipp! 🥳

@marimeireles marimeireles merged commit 829cc9f into main Jun 21, 2022
2 checks passed
@marimeireles marimeireles deleted the pytest_playwright branch June 21, 2022 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tag: tests Related to tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants