Skip to content

#12392 Add benchmark for overheads of trial test runner.#12393

Merged
adiroiban merged 2 commits intotwisted:trunkfrom
p12tic:benchmark-trial-testcase
Dec 9, 2024
Merged

#12392 Add benchmark for overheads of trial test runner.#12393
adiroiban merged 2 commits intotwisted:trunkfrom
p12tic:benchmark-trial-testcase

Conversation

@p12tic
Copy link
Contributor

@p12tic p12tic commented Dec 9, 2024

Scope and purpose

Fixes #12392

@codspeed-hq
Copy link

codspeed-hq bot commented Dec 9, 2024 •
edited
Loading

CodSpeed Performance Report

Merging #12393 will not alter performance

Comparing p12tic:benchmark-trial-testcase (0d2d3af) with trunk (3679783)

Summary

✅ 25 untouched benchmarks
🆕 9 new benchmarks

Benchmarks breakdown

Benchmark trunk p12tic:benchmark-trial-testcase Change
🆕 test_main_method[testAddCleanupPlain] N/A 1.1 ms N/A
🆕 test_main_method[testAddCleanupResolvedDeferred] N/A 1.1 ms N/A
🆕 test_main_method[testAddCleanupUnresolvedDeferred] N/A 1.4 ms N/A
🆕 test_main_method[testPlain] N/A 1.2 ms N/A
🆕 test_main_method[testResolvedDeferred] N/A 1.1 ms N/A
🆕 test_main_method[testUnresolvedDeferred] N/A 1.4 ms N/A
🆕 test_setup_teardown[BenchmarkTestsSetUpTearDownPlain] N/A 1.1 ms N/A
🆕 test_setup_teardown[BenchmarkTestsSetUpTearDownResolvedDeferred] N/A 1.1 ms N/A
🆕 test_setup_teardown[BenchmarkTestsSetUpTearDownUnresolvedDeferred] N/A 1.4 ms N/A

@p12tic p12tic mentioned this pull request Dec 9, 2024
Merged
@p12tic p12tic force-pushed the benchmark-trial-testcase branch from 3593e30 to f2964ab Compare December 9, 2024 12:13
@p12tic
Copy link
Contributor Author

p12tic commented Dec 9, 2024

please review

@chevah-robot chevah-robot requested a review from a team December 9, 2024 12:14
@p12tic p12tic mentioned this pull request Dec 9, 2024
Merged
adiroiban
adiroiban approved these changes Dec 9, 2024
View reviewed changes
Copy link
Member

@adiroiban adiroiban left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I think that this is a good start and we can merge it.

We will see later if these tests are flaky or not.

@p12tic p12tic force-pushed the benchmark-trial-testcase branch from f2964ab to 7b5ab7c Compare December 9, 2024 15:46
@p12tic
Copy link
Contributor Author

p12tic commented Dec 9, 2024

@adiroiban I fixed a static check problem, should be ready to merge.

@adiroiban
Copy link
Member

adiroiban commented Dec 9, 2024 •
edited
Loading

It looks like the test_tcp_throughput test is flaky ... but this is not related to the changes from this PR.

image

I triggerred a re-run of the benchmarks tests and it looks ok.

There is this new warning ... looks like pytest bug.

Maybe we should just add the __test__ = False class member to TestResult to make pytest happy.

    /opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages/twisted/trial/reporter.py:83: PytestCollectionWarning: cannot collect test class 'TestResult' because it has a __init__ constructor (from: benchmarks/test_trial.py)
      @implementer(itrial.IReporter)

@adiroiban adiroiban enabled auto-merge December 9, 2024 19:04
@adiroiban adiroiban disabled auto-merge December 9, 2024 19:04
@adiroiban adiroiban changed the title Add benchmark for overheads of trial test runner (#12392) #12392 Add benchmark for overheads of trial test runner Dec 9, 2024
@adiroiban adiroiban changed the title #12392 Add benchmark for overheads of trial test runner #12392 Add benchmark for overheads of trial test runner. Dec 9, 2024
@adiroiban adiroiban enabled auto-merge December 9, 2024 19:04
@adiroiban
Copy link
Member

adiroiban commented Dec 9, 2024

Thanks again for the update. I have enabled the auto-merge. I hope it will land soon.

@adiroiban adiroiban merged commit 8062f3e into twisted:trunk Dec 9, 2024
@p12tic p12tic deleted the benchmark-trial-testcase branch December 17, 2024 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@adiroiban adiroiban adiroiban approved these changes

+1 more reviewer

@Ameerjb1993 Ameerjb1993 Ameerjb1993 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

needs-merge

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

twisted.trial.unittest.TestCase should have a benchmark

4 participants

@p12tic @adiroiban @Ameerjb1993 @chevah-robot