Add CodSpeed benchmark suite for parser hot paths#276
Merged
Conversation
Congrats! CodSpeed is installed 🎉
You will start to see performance impacts in the reports once the benchmarks are run from your default branch.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b7241a651f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Five benchmarks in tests/test_benchmarks.py covering the parsers Starlette actually uses (MultipartParser and QuerystringParser): - multipart small form (typical login-style fields) - multipart file upload (2 MiB file in 64 KiB chunks) - multipart mixed form (fields plus file in chunked stream) - urlencoded small form - urlencoded large form fragmented Adds pytest-codspeed as a dev dep and a CodSpeed workflow that runs in instrumentation mode on push to main and on pull requests.
b7241a6 to
f1e3395
Compare
mypy --strict rejected the bare dict literal because the noop callable did not unify with the typed callback signatures. Splitting noop into event-shaped and data-shaped helpers and annotating the dicts with the exported TypedDicts satisfies the checker.
Inspired by defnull/multipart_bench: long random-shaped boundary matching what browsers emit, varied payload bytes via string.printable, and a worstcase scenario where the body contains partial-boundary character matches to exercise the find-miss tail scan. Five scenarios: simple form, 100-field form, 8 MiB file upload, worstcase boundary chars, and a 100-field urlencoded form.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests/benchmarks/coveringMultipartParser,FormParser,parse_form,OctetStreamParser, andQuerystringParserpytest-codspeedto the dev dependency group.github/workflows/benchmark.ymlrunning CodSpeed in instrumentation mode on push tomainand on pull requestsWhy
Without continuous benchmarking we have no signal on parser performance regressions across PRs. This wires up CodSpeed to track parsing latency on the hot paths we actually care about.
The structure mirrors uvicorn's CodSpeed setup: a benchmarks directory under
tests/, a top-levelpytest.mark.benchmarkmarker on each module, and a workflow that runspytest tests/benchmarks/ --codspeedviaCodSpeedHQ/action.Coverage of scenarios:
MultipartParser: single-chunk parse, fragmented (64 B chunks), 2 MiB file streamed in 16 KiB chunks, parametrized partial-boundary scan across boundary lengths (16, 64, 256, 1024 B), many small parts.FormParser: multipart and urlencoded paths.parse_form: parametrized across body sizes (1 KiB, 64 KiB, 1 MiB).OctetStreamParserandQuerystringParser: streamed and fragmented.The benchmarks also pass as plain pytest tests (no
--codspeed), so they run alongside the regular suite. Coverage stays at 100% (tests/*is already excluded).Test plan
pytest tests/benchmarks/passes locally (16 tests)pytest tests/benchmarks/ --codspeedruns cleanly in walltime mode locallyscripts/teststill hits 100% coverage (157 tests total)AI Disclaimer
This PR was developed with the assistance of either Claude or Codex. I've reviewed and verified the changes.