Skip to content

[EPIC] Improve test coverage #679

@antocuni

Description

@antocuni

This is an open ended issue to encourage to add more tests to improve our coverage.
We recently did a lot of work to improve our test situation:

PRs to add more tests are very welcome. For new contributors, it's probably a good idea to start by writing integration tests, since they are the "most ready". A good starting point is to look at the (few) existing tests:

def test_pyscript_hello(self):
self.pyscript_run(
"""
<py-script>
print('hello pyscript')
</py-script>
"""
)
# this is a very ugly way of checking the content of the DOM. If we
# find ourselves to write a lot of code in this style, we will
# probably want to write a nicer API for it.
inner_html = self.page.locator("py-script").inner_html()
pattern = r'<div id="py-.*">hello pyscript</div>'
assert re.search(pattern, inner_html)

And the docs for the testing machinery itself:

class PyScriptTest:
"""
Base class to write PyScript integration tests, based on playwright.
It provides a simple API to generate HTML files and load them in
playwright.
It also provides a Pythonic API on top of playwright for the most
common tasks; in particular:
- self.console collects all the JS console.* messages. Look at the doc
of ConsoleMessageCollection for more details.
- self.check_errors() checks that no JS errors have been thrown
- after each test, self.check_errors() is automatically run to ensure
that no JS error passes uncaught.
- self.wait_for_console waits until the specified message appears in the
console
- self.wait_for_pyscript waits until all the PyScript tags have been
evaluated
- self.pyscript_run is the main entry point for pyscript tests: it
creates an HTML page to run the specified snippet.
"""

The list of things which we would like to test includes, but it's not limited to:

  • the behavior of <py-config> and <py-env>
  • the behavior of event handlers (like e.g. test_py_button.py:test_click)
  • the behavior of sync vs async <py-script>
  • the ability of generating HTML from a <py-script> tag
  • the behavior of the various <py-*> components

Ideally, a good test should have the following properties:

  • it should be short and self contained; even if they are integration test, we want to test one feature at a time
  • it should not be fragile and flaky: the content of dynamic web pages is changing continuously so it's hard to write robust tests; playwright auto waiting should help in this regard.

Moreover, we also want to improve the tests for our examples. Once #676 is merged, there will be many XXX improve this test inside test_zz_examples.py, e.g.:
https://github.com/pyscript/pyscript/blob/513dfe0b428c70406d40eef3d9851dabdf27bca6/pyscriptjs/tests/test_zz_examples.py#L76-L81

Metadata

Metadata

Assignees

No one assigned

    Labels

    epica highlevel collection of smaller related issueshelp wantedExtra attention is neededlang: pythonRelated to the Python languagetag: componentRelated to PyScript componentstag: exampleRelated to adding or improving examplestag: testsRelated to teststype: featureNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    Closed

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions