Skip to content

The integration test 'test_repl' is flaky #726

@FabioRosado

Description

@FabioRosado

Checklist

  • I added a descriptive title
  • I searched for other issues and couldn't find a duplication
  • I already searched in Google and didn't find any good information or help

What is the issue/comment/problem?

While running the integration tests for the examples, all of them seem to pass locally, but in CI the test_repl seems to fail very often. I've marked this test as xfail in #711 and this issue is mostly to track that.

From the CI:

=================================== FAILURES ===================================
[3261](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3262)
_______________________ TestExamples.test_repl[chromium] _______________________
[3262](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3263)

[3263](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3264)
self = <integration.test_zz_examples.TestExamples object at 0x7f03c7dc7640>
[3264](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3265)

[3265](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3266)
    def test_repl(self):
[3266](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3267)
        self.goto("examples/repl.html")
[3267](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3268)
        self.wait_for_pyscript()
[3268](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3269)
        assert self.page.title() == "REPL"
[3269](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3270)
        wait_for_render(self.page, "*", "<py-repl.*?>")
[3270](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3271)
    
[3271](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3272)
        self.page.locator("py-repl").type("print('Hello, World!')")
[3272](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3273)
        self.page.locator("button").click()
[3273](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3274)
    
[3274](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3275)
        assert self.page.locator("#my-repl-1").text_content() == "Hello, World!"
[3275](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3276)
    
[3276](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3277)
        # Confirm that using the second repl still works properly
[3277](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3278)
        self.page.locator("#my-repl-2").type("2*2")
[3278](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3279)
        self.page.keyboard.press("Shift+Enter")
[3279](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3280)
    
[3280](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3281)
>       assert self.page.locator("#my-repl-2-2").text_content() == "4"
[3281](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3282)
E       AssertionError: assert '' == '4'
[3282](https://github.com/pyscript/pyscript/runs/8026073420?check_suite_focus=true#step:8:3283)
E         - 4

It's interesting that only the second check is failing, which might mean that perhaps the keyboard press didn't work or we have to wait a bit for it to show up.

I'll try to look into this tomorrow 😄

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    Closed

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions