Conversation
6889fce to
412bc3d
Compare
Lost commits from 4bd79e3...0e4ae38 in the squash. This readds them.
|
|
||
|
|
||
| def request(method, url, **kwargs): | ||
| def request(method: str, url: str, **kwargs: Any) -> Response: |
There was a problem hiding this comment.
I know it would be a lot of work, but annotating the kwargs with an Unpack[KwargsDict] where KwargsDict is a typing.TypedDict would be great!
I hacked this into my own code that interacts with requests code to make my usage more safe:
- where I defined my own typed dict class https://github.com/cthoyt/pystow/blob/1ca7fa5763c6aa0d2138a164c78fd67955152ebc/src/pystow/utils/download.py#L63-L72
- where I annotate it on a function I wrote, that wraps
requsts.get: https://github.com/cthoyt/pystow/blob/1ca7fa5763c6aa0d2138a164c78fd67955152ebc/src/pystow/utils/download.py#L87 - where the kwargs get splatted into
requests.get: https://github.com/cthoyt/pystow/blob/1ca7fa5763c6aa0d2138a164c78fd67955152ebc/src/pystow/utils/download.py#L165
There was a problem hiding this comment.
also I am willing to PR this if you think it's a good idea / if you think an external contrib is better/easier than doing it yourself
There was a problem hiding this comment.
Thanks for comment, @cthoyt! This is a good callout because it is a delta from typeshed that should've been in the overview.
I played with Unpack for this but put that on pause because it got kind of messy. From what I was able to come up with we're going to need several different TypedDicts to accomplish it. Essentially one per signature, it's definitely doable, just clunky.
If you have ideas to simplify the approach, I'd love to hear more. This is something that needs an answer before we ship.
There was a problem hiding this comment.
one thing worth noting is that you can have (multiple) inheritance on typed dicts. this can make it easier to factor out similarities between typed dict definitions for several functions
| _ParamsMappingValueType: TypeAlias = ( | ||
| str | bytes | int | float | Iterable[str | bytes | int | float] | None | ||
| ) | ||
| ParamsType: TypeAlias = ( |
There was a problem hiding this comment.
it would be lovely if these type hints were available through a "public" interface, since I've often written code that wraps a requests call, where I would want to use the ParamsType to annotate my own code
I say public in quotes because I could import this, but it feels wrong
There was a problem hiding this comment.
I agree having some form of public hints for the top-level APIs is potentially warranted. I started very conservative here because often times new code goes into Requests and immediately comes someone else's critical dependency. I don't want to create a binding contract that people start surfacing in their code, and then "break" when we need to update/tweak it.
I think once we're really happy with the types, the main argument parameters could be surfaced. I've deferred that for now until we can make a more informed decision. Presumably everything calling in is already typed sufficiently.
There was a problem hiding this comment.
fair! thank you for doing this and considering carefully :)
| method=request.method, | ||
| url=url, | ||
| body=request.body, | ||
| body=request.body, # type: ignore[arg-type] # urllib3 stubs don't accept Iterable[bytes | str] |
There was a problem hiding this comment.
Should be addressed with urllib3/urllib3#3799.
srittau
left a comment
There was a problem hiding this comment.
Thanks for adding type hints! As a typeshed maintainer, I'm always glad to see annotations being added directly to packages. Less work for me , but more importantly as better user experience.
Also a general note: Some kwargs arguments could probably be accurately annotated using Unpack[_SomeTypedDict]. See for example the ast.pyi module in typeshed.
You could also consider marking constants as Final. This sometimes even makes type annotations easier. For example from below: Instead of
DEFAULT_PORTS: dict[str, int] = {"http": 80, "https": 443}you could use
DEFAULT_PORTS: Final = {"http": 80, "https": 443}At mypy infers the same type, although in some cases type checkers can even infer literals.
Some specific suggestions below.
| - name: Set up Python | ||
| uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 | ||
| with: | ||
| python-version: "3.10" |
There was a problem hiding this comment.
You could consider using a version matrix here. This could flag removals, API changes, and deprecations in CPython's stdlib early. Maybe just using the oldest and latest CPython version supported by requests would be enough as a smoke test.
There was a problem hiding this comment.
Yeah, that's fair. I started with the minimum because we've generally tested floors but I agree having something for the latest version also makes sense to catch breakages as we add support.
| @runtime_checkable | ||
| class SupportsRead(Protocol[_T_co]): | ||
| def read(self, length: int = ...) -> _T_co: ... |
There was a problem hiding this comment.
You could consider conditionally using io.Reader when using Python 3.14+. It is nearly identically defined, with one difference (pos-only length argument) that I would suggest adding:
| @runtime_checkable | |
| class SupportsRead(Protocol[_T_co]): | |
| def read(self, length: int = ...) -> _T_co: ... | |
| @runtime_checkable | |
| class SupportsRead(Protocol[_T_co]): | |
| def read(self, length: int = ..., /) -> _T_co: ... |
| HookType = Callable[["Response"], Any] | ||
| HooksInputType = Mapping[str, "Iterable[HookType] | HookType"] |
There was a problem hiding this comment.
Shouldn't these be marked as TypeAliases?
There was a problem hiding this comment.
Yep, also a good call out. I'll get those added.
|
|
||
| TimeoutType: TypeAlias = float | tuple[float | None, float | None] | None | ||
| ProxiesType: TypeAlias = MutableMapping[str, str] | ||
| HooksType: TypeAlias = dict[str, list["HookType"]] | None |
There was a problem hiding this comment.
Nit: HookType doesn't need to be quotes as it's defined before in this file.
There was a problem hiding this comment.
Good callout, the Hook* types were originally at the bottom of the file, but got lifted just before I posted this.
|
Really appreciate the response and your time looking at this, @srittau! I can get most of those addressed quick. The unpack piece I covered quickly with Chris in #7272 (comment). That's definitely a delta from typeshed we'll want to cover. The initial attempt on that got messy pretty quickly so I sidelined it. We'll get that addressed before shipping this though. |
|
I've added https://gist.github.com/nateprewitt/3e65fc8c3e9db3d5629ec1daf1f993f8 to track our current checklist before merging this. |
Add inline type annotations
Important
We want feedback from people who actively maintain projects that depend on Requests or use it heavily. Please share your experience testing this against your code in the linked issue.
Comments that are clearly AI-generated will be hidden or removed. This is a library written "for Humans". The conversation is between maintainers and users, not a prompt.
Overview
This PR adds type annotations throughout Requests to codify the type contract alongside the code. This is a path for what inline typing in Requests could look like. The goals are to establish what APIs are intended to support, make more reasoned decisions about code changes, and make using Requests in a modern development environment easier.
The package passes strict type checking with pyright, with the caveat that a few decisions on internals have been deferred for discussion by the maintainers and broader community. Right now, the intended behavior for those is undefined.
Related issue: RFC: Adding inline type annotations to Requests
Currently tracked TODOs: https://gist.github.com/nateprewitt/3e65fc8c3e9db3d5629ec1daf1f993f8
How to verify locally
Should report 0 errors.
typing-extensionsis needed on Python 3.10-3.12.Main design decisions
cast()overassert/isinstancefor type narrowing to avoid breaking downstream code at runtime_types.pymodule centralizing Protocols, type aliases, and TypeVarsSupportsRead/SupportsItemsas@runtime_checkableProtocols replacinghasattrchecksis_prepared()as aTypeIsguard that's a no-op at runtime (always returnsTrue)Runtime impact
A full catalog of every runtime-impacting change is available here. The behavioral changes and bug fixes are the ones worth scrutinizing. The rest is mechanical.
Open discussion points
strvsbytesURL handling: Requests has historically accepted both when Python 2.7 was primary usage. These types currently declarestronly. This needs a decision on whether to fix existing broken bytes behavior or formalize thestr-only contract.yield_requestsAPI: Needs documentation and a decision on how this should even work in practice. This API decision came through without review and I don't think there's any realistic usage outside of the library.What's NOT in this PR