I must say that I don't find the plug-in-less output (example here) less readible than the previously-working one (example here). Additionally, the summary thingy works quite nicely IMO: https://github.com/python-telegram-bot/python-telegram-bot/actions/runs/5369863913#summary-14534142641

tbh I don't like seeing the big coverage report table in the output, but i'll let that slide since I really like the summary view. Btw the counter for the tests is a bit wrong?

Btw the counter for the tests is a bit wrong?

Mh, that could be because I generated the failure report only on the "run last failures" run 🤔

Btw the counter for the tests is a bit wrong?

Reading the code of the action it seems like they for the XML report

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
    <testsuite errors="0" failures="1" hostname="fv-az1252-633" name="pytest" skipped="0" tests="1"
               time="8.881" timestamp="2023-06-25T20:55:18.083387">
        <testcase classname="tests.request.test_request.TestHTTP2WithRequest"
                  name="test_http_2_response" time="0.004"/>
        <testcase classname="tests.request.test_request.TestHTTP2WithRequest"
                  name="test_http_2_response" time="0.003"/>
        <testcase classname="tests.request.test_request.TestHTTP2WithRequest"
                  name="test_http_2_response" time="0.050">
            <failure
                    message="AssertionError: assert 'HTTP/2' == 'HTTP/1'&#10;  - HTTP/1&#10;  ?      ^&#10;  + HTTP/2&#10;  ?      ^">
                self = &lt;tests.request.test_request.TestHTTP2WithRequest object at
                0x000001CD3B08BF08&gt;

                async def test_http_2_response(self):
                httpx_request = HTTPXRequest(http_version="2")
                async with httpx_request:
                resp = await httpx_request._client.request(
                url="https://python-telegram-bot.org",
                method="GET",
                headers={"User-Agent": httpx_request.USER_AGENT},
                )
                &gt; assert resp.http_version == "HTTP/1"
                E AssertionError: assert 'HTTP/2' == 'HTTP/1'
                E - HTTP/1
                E ? ^
                E + HTTP/2
                E ? ^

                tests\request\test_request.py:96: AssertionError
            </failure>
        </testcase>
    </testsuite>
</testsuites>

the number of failures and tests is aggregated by parsing each testcase entry rather than parsing the header of the testuite entry. Because tests are flaky, this sums up to 2 successfull & 1 failed test, which is probably not the result that we expect. Plus, because the reports are generated only on the --lf run, it doesn't include the large number of successful tests at all.

An alternative output strategy would be to also include the results of the first run, but this leads to failures that repeat in the --lf run to be reported twice. IMO the number of successful tests is not overly interesting anyways, only the number of failed tests. So personally I don't really care for the summary SVG and I'm good with ignoring this shortcuming.

I must say that I don't find the plug-in-less output (example here) less readible than the previously-working one (example here). Additionally, the summary thingy works quite nicely IMO: https://github.com/python-telegram-bot/python-telegram-bot/actions/runs/5369863913#summary-14534142641

I agree that there is not much difference and that the summary view is very nice

@lemontree210 @harshil21 may I understand your comments as approval, then? :)

@lemontree210 @harshil21 may I understand your comments as approval, then? :)

mine yes, though not a very competent one 🙄