Skip to content

Add no-backend equivalent of test_several_tools_add_and_remove benchmark#1856

Merged
nathanjmcdougall merged 3 commits into
mainfrom
copilot/create-benchmark-mocking-uv
Apr 7, 2026
Merged

Add no-backend equivalent of test_several_tools_add_and_remove benchmark#1856
nathanjmcdougall merged 3 commits into
mainfrom
copilot/create-benchmark-mocking-uv

Conversation

Copilot AI commented Apr 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

test_several_tools_add_and_remove is slow because it subprocesses uv. This adds a directly comparable benchmark that skips uv entirely using --backend=none, enabling meaningful performance comparisons between the two.

Changes

  • New benchmark test_several_tools_add_and_remove_no_backend in tests/usethis/_ui/interface/test_tool.py — identical tool sequence as the original but with --backend none on every invocation (~0.2s vs ~5s)
  • Fix pre-existing check-agent-file-sizes violation — AGENTS.md was 514 lines against a 500-line limit; added --skip-private to the export-functions hook so only public-API functions appear in the agent reference (docs/functions.txt: 233→196 lines, AGENTS.md: 514→477 lines)
@pytest.mark.benchmark
def test_several_tools_add_and_remove_no_backend(tmp_path: Path):
    ...
    with change_cwd(tmp_path):
        result = runner.invoke_safe(app, ["pytest", "--backend", "none"])
        result = runner.invoke_safe(app, ["coverage", "--backend", "none"])
        result = runner.invoke_safe(app, ["ruff", "--backend", "none"])
        result = runner.invoke_safe(app, ["deptry", "--backend", "none"])
        result = runner.invoke_safe(app, ["pre-commit", "--backend", "none"])
        result = runner.invoke_safe(app, ["ruff", "--remove", "--backend", "none"])
        result = runner.invoke_safe(app, ["pyproject-fmt", "--backend", "none"])
        result = runner.invoke_safe(app, ["pytest", "--remove", "--backend", "none"])

Copilot AI linked an issue Apr 7, 2026 that may be closed by this pull request
Create an equivalent benchmark to test_several_tools_add_and_remove which mocks uv #378
Closed
@nathanjmcdougall
Add test_several_tools_add_and_remove_no_backend benchmark using --ba…
8002985 
…ckend=none; fix AGENTS.md line count by skipping private functions in export-functions hook

Agent-Logs-Url: https://github.com/usethis-python/usethis-python/sessions/25183865-1b63-43aa-b3a1-1cc7fac1091e

Co-authored-by: nathanjmcdougall <18602289+nathanjmcdougall@users.noreply.github.com>
Copilot AI changed the title [WIP] Create benchmark to mock uv for comparison Add no-backend equivalent of test_several_tools_add_and_remove benchmark Apr 7, 2026
Copilot AI requested a review from nathanjmcdougall April 7, 2026 10:45
@nathanjmcdougall nathanjmcdougall marked this pull request as ready for review April 7, 2026 11:00
@codecov

codecov Bot commented Apr 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Apr 7, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 2 untouched benchmarks
🆕 1 new benchmark

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 Simulation test_several_tools_add_and_remove_no_backend N/A 467 ms N/A

Comparing copilot/create-benchmark-mocking-uv (4b68fa8) with main (b868419)

Open in CodSpeed

@nathanjmcdougall nathanjmcdougall merged commit 813b338 into main Apr 7, 2026
20 checks passed
@nathanjmcdougall nathanjmcdougall deleted the copilot/create-benchmark-mocking-uv branch April 7, 2026 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@nathanjmcdougall nathanjmcdougall nathanjmcdougall approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@nathanjmcdougall nathanjmcdougall

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Create an equivalent benchmark to test_several_tools_add_and_remove which mocks uv

2 participants

@nathanjmcdougall