Skip to content

fix(x509): use backward compatible syntax for types#12927

Merged
alex merged 2 commits intopyca:mainfrom
shcheklein:fix/x509-python-39-types
May 17, 2025
Merged

fix(x509): use backward compatible syntax for types#12927
alex merged 2 commits intopyca:mainfrom
shcheklein:fix/x509-python-39-types

Conversation

@shcheklein
Copy link
Contributor

@shcheklein shcheklein commented May 17, 2025

An attempt to fix the issue we are hitting on mypy Python 3.9 (but also I think it won't work with Python 3.9 itself as well):

.nox/lint/lib/python3.9/site-packages/cryptography/x509/certificate_transparency.py:8: note: In module imported here,
.nox/lint/lib/python3.9/site-packages/cryptography/x509/__init__.py:7: note: ... from here,
.nox/lint/lib/python3.9/site-packages/werkzeug/serving.py:93: note: ... from here,
.nox/lint/lib/python3.9/site-packages/werkzeug/__init__.py:1: note: ... from here:
.nox/lint/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust/x509.pyi:232:7: error:
invalid syntax  [syntax]
    type MaybeExtensionValidatorCallback[T: x509.ExtensionType] = typing.C...
          ^
Found 1 error in 1 file (errors prevented further checking)

nox > Command pre-commit run --show-diff-on-failure --all-files failed with exit code 1
nox > Session lint failed.

It was introduced originally here: 4cfd7b1

I would appreciate a review since I'm definitely not a Python types expert.

@shcheklein shcheklein force-pushed the fix/x509-python-39-types branch from c9b98db to 05f3ded Compare May 17, 2025 20:36
@shcheklein shcheklein marked this pull request as ready for review May 17, 2025 20:46
@shcheklein
Copy link
Contributor Author

@deivse @alex @reaperhulk I would appreciate a review / suggestions when you a minute. Thanks 🙏

@deivse
Copy link
Contributor

deivse commented May 17, 2025

I remember thinking if this is fine when making the original PR, yeah... I think we agreed during review that using the newer syntax shouldn't cause any issues, since it's an interface file, but it actually can, as I have encountered since. Normally mypy won't complain about this, but if you're running with min python version specified in mypy config (I think it can also pick it up from pyproject.toml), it will. I assume you have the min python version explicitly specified somewhere?

Fortunately this should not be an issue outside of type checking - the .pyi file is purely for interface definitions and type checking information. I definitely agree this should be fixed though.

Copy link
Member

@alex alex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a job to ci.yml, probably something like:

          - {VERSION: "3.8", NOXSESSION: "flake"}

@alex alex linked an issue May 17, 2025 that may be closed by this pull request
NON_CRITICAL: Criticality

type MaybeExtensionValidatorCallback[T: x509.ExtensionType] = typing.Callable[
T = typing.TypeVar("T", covariant=True, bound=x509.ExtensionType)
Copy link
Contributor

@deivse deivse May 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should actually be contravariant, not covariant - this doesn't affect usage in the method definitions, but for the type aliases, since they are callables, we want them to be contravariant, I think. Please correct me if I'm wrong - my first encounter with covariance and friends.

From what I understood, making this covariant means that wherever a MaybeExtensionValidator[A] is accepted, a MaybeExtensionValidator[B] would also be accepted, where B is derived from A. But I don't think that's a good idea. Contravariance would mean that MaybeExtensionValidator[A] would be accepted wherever MaybeExtensionValidator[B] is accepted, and that seems potentially useful in some specific cases - i.e. for example the user might want to create some extension validation callback that accepts any type (type hint would be for x509.ExtensionType) and re-routes the validation to some other structure in their code.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mypy docs describe why contravariance makes sense for callables better than I do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right. so, do you feel we need to put explicit contravariant=True or leave it default?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explicitly mark contravariant I think, because with the old type syntax it's neither by default, and the new one defaulted this to contravariant. I think it's best to keep it that way, since the use case of passing in an extension validator that accepts any extension can be valid under some very specific circumstances (and I'm pretty sure I did that in tests).

@alex can you please double check that my understanding of co/contravariance is correct when you have a chance? 😄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think leaving it blank makes it invariant, and that's fine?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imo that's a bit needlessly prohibitive since then you wouldn't be able to pass in an extension validator that accepts any extension. I'm pretty sure that would make type checking fail for the tests. The real world use cases I can come up with are extremely contrived, but I think it's worth keeping it contravariant just for the tests since it doesn't hurt anything imo.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about the tests tbh, basically either nothing or explicitly contravariant should be fine. Not a deal breaker imo.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that inavariant is the same behavior as the prior type annotation:

>>> def f[T: datetime.datetime](t: T): pass
... 
>>> f.__annotations__
{'t': T}
>>> t = f.__annotations__["t"]
>>> t.__covariant__
False
>>> t.__contravariant__
False

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is different since it's a function accepting a type, there any sort of variance is not defined since the function itself is not generic, it just accepts a generic argument. If you defined a type alias for a callable with the new syntax I'm quite sure it would be contravariant. (Or alternatively a function which accepts a generic callable.)

The idea is that when u accept a callable you want to accept a callable for a specific type or a callable for a more generic type - all you care about is that you can call it with your type.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(at least mypy docs explicitly say that generic callables are contravariant by default with the new syntax)

@shcheklein
Copy link
Contributor Author

@alex added, seems there are other issue on Python 3.8, probably unrelated to this one.

@alex
Copy link
Member

alex commented May 17, 2025

Grumble. Ok I think what I'll do is send a separate PR to resolve those, so you don't have to debug all of our (apparently broken) type annotations.

alex added a commit to alex/cryptography that referenced this pull request May 17, 2025
@shcheklein shcheklein force-pushed the fix/x509-python-39-types branch 2 times, most recently from 05f3ded to 0e72897 Compare May 17, 2025 21:59
reaperhulk pushed a commit that referenced this pull request May 17, 2025
@alex
Copy link
Member

alex commented May 17, 2025

Ok you should be able to merge/rebase main and get green.

@shcheklein shcheklein force-pushed the fix/x509-python-39-types branch from 0e72897 to 23b0ee4 Compare May 17, 2025 22:08
@alex
Copy link
Member

alex commented May 17, 2025

(And we need to bring back the 3.8 flake job)

@alex
Copy link
Member

alex commented May 17, 2025

doh. missed one #12929

@shcheklein
Copy link
Contributor Author

@alex still getting some issues on Py 3.8:

nox > mypy src/cryptography/ vectors/cryptography_vectors/ tests/ release.py noxfile.py
release.py:10: error: Cannot find implementation or library stub for module named "tomllib"  [import-not-found]
release.py:10: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 205 source files)
pyproject.toml: [mypy]: Unrecognized option: strict_bytes = True
nox > Command mypy src/cryptography/ vectors/cryptography_vectors/ tests/ release.py noxfile.py failed with exit code 1

@shcheklein
Copy link
Contributor Author

I'll rebase when it is merged ...

@alex
Copy link
Member

alex commented May 17, 2025

Ok, it's merged

@shcheklein shcheklein force-pushed the fix/x509-python-39-types branch from 1d23154 to ea374dc Compare May 17, 2025 22:30
@shcheklein shcheklein force-pushed the fix/x509-python-39-types branch from ea374dc to fd3860a Compare May 17, 2025 22:35
@alex alex enabled auto-merge (squash) May 17, 2025 22:39
@alex alex merged commit 1be08b9 into pyca:main May 17, 2025
65 checks passed
alex added a commit to alex/cryptography that referenced this pull request May 17, 2025
alex pushed a commit to alex/cryptography that referenced this pull request May 17, 2025
* fix(x509): use backward compatible syntax for types

* add Python 3.8 flake to ci
reaperhulk pushed a commit that referenced this pull request May 18, 2025
* mypy cleanups for Python 3.8 (#12928)

should unblock #12927

* fix typing on release.py with older python (#12929)

* fix(x509): use backward compatible syntax for types (#12927)

* fix(x509): use backward compatible syntax for types

* add Python 3.8 flake to ci

* Backport mypy fixes for release

* sigh

* sigh

---------

Co-authored-by: Ivan Shcheklein <shcheklein@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Broken typing file in cryptography 45.0.1

3 participants