Skip to content

Conversation

@ntBre
Copy link
Contributor

@ntBre ntBre commented Apr 24, 2025

Summary

This PR resolves #9761 by adding a linter configuration option to disable
typing_extensions imports. As mentioned here, it would be ideal if we could
detect whether or not typing_extensions is available as a dependency
automatically, but this seems like a much easier fix in the meantime.

The default for the new option, typing-extensions, is true,
preserving the current behavior. Setting it to false will bail out of the new
Checker::typing_importer method, which has been refactored from the
Checker::import_from_typing method in #17340),
with None, which is then handled specially by each rule that calls it.

I considered some alternatives to a config option, such as checking if typing_extensions has been imported or checking for a TYPE_CHECKING block we could use, but I think defaulting to allowing typing_extensions imports and allowing the user to disable this with an option is both simple to implement and pretty intuitive.

Test Plan

New linter tests exercising several combinations of Python versions and the new config option for PYI019. I also added tests for the other affected rules, but only in the case where the new config option is enabled. The rules' existing tests also cover the default case.

ntBre added 12 commits April 24, 2025 10:23
Summary
--

This PR resolves #9761 by adding a linter configuration option to disable
`typing_extensions` imports. As mentioned [here], it would be ideal if we could
detect whether or not `typing_extensions` is available as a dependency
automatically, but this seems like a much easier fix in the meantime.

The default for the new option `disable-typing-extensions` is `false`,
preserving the current behavior. Setting it to `true` will bail out of the new
`Checker::import_from_typing` method (#17340) with a new
`TypingImportError::TypingExtensionsDisabled` error, which should log an
explanatory message whenever `Diagnostic::try_set_fix` is used.

[here]: #9761 (comment)

Test Plan
--

New linter tests exercising several combinations of Python versions and the new
config option for PYI019. We could also test other rules, but any rule using
`import_from_typing` should behave the same way.
this has more to do with the python version than the config option. changing the
option doesn't actually have any effect here. the version must be validated
somewhere else along the way
@ntBre ntBre added rule Implementing or modifying a lint rule configuration Related to settings and configuration labels Apr 24, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Apr 24, 2025

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

@ntBre ntBre marked this pull request as ready for review April 24, 2025 16:02
@ntBre ntBre requested a review from AlexWaygood as a code owner April 24, 2025 16:02
@ntBre ntBre requested a review from MichaReiser April 25, 2025 15:27
Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I agree with you, the way typing_import works today is a bit awkward where it's very easy to omit the fix instead of omitting the diagnostic. I added an inline suggestion on what you could try to improve the ergonomics.

Comment on lines 548 to 553
pub(crate) fn import_from_typing(
&self,
member: &str,
position: TextSize,
version_added_to_typing: PythonVersion,
) -> Result<(Edit, String), ResolutionError> {
) -> Result<Option<(Edit, String)>, ResolutionError> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could use a pattern similar to @BurntSushi's InferContext::report_lint where the inital function returns an Option<Builder> and the builder allows you to perform the typing import.

pub(crate) fn typing_importer(&self, version_added_to_typing: PythonVersion) -> Option<TypingImporte> {
	let source_module = if self.target_version() >= version_added_to_typing {            "typing"
	} else if self.settings.disable_typing_extensions {
  	return None;
	} else {
		"typing_extensions"
	};
	
	Some(TypingImporter { checker: self, source_module })
}

pub struct TypingImporter<'a> {
	checker: &'a Checker,
	source_module: &'static src
}

impl TypingImporter<'_> {
	pub fn import(member: &str, position: TextSize) -> Result<(Edit, String), ResolutionError> {
		... 
	}
}

I'm a bit undecided if member should be passed to typing_importer to avoid accidential reuse.

A simpler alternative would be to return Option<Result<...>, but that's a bit less common but could be fine too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I like that idea, I'll give it a try!

I vaguely remember trying Option<Result> and not liking it for some reason, but maybe that was when I thought try_set_optional_fix was still going to work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we do want member passed to typing_importer because it's the piece that needs to be tied most closely to the PythonVersion. I don't remember seeing any places where intentional reuse would be very helpful, so avoiding the accidental cases makes sense to me.

/// parameters to fix the issue. In the latter case, the lint is unactionable because
/// `typing_extensions` imports have been explicitly disallowed, and we should return without a
/// `Diagnostic`.
enum Fixable {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to update the rule's fix availability documentation (same for other rules) to explain that the fix isn't available if typing extensions isn't available?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, very good point, thanks!

disable-typing-extensions = true
"#
)]
pub disable_typing_extensions: Option<bool>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bikeshedding: The disable prefix is uncommon in our existing settings and simply typing_extensions would be more idiomatic or allow_typing_extensions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I always like options that default to false so that I can use bool::default 😆 but either of those names is fine with me

@ntBre
Copy link
Contributor Author

ntBre commented Apr 28, 2025

Thanks again for the suggestions! I've updated the code to:

  • use your TypingImporter idea
  • add an ## Availability section to each affected rule
    • I called it Availability instead of the more typical Fix availability because it affects the diagnostic too
    • I left out any rules using PythonVersion::lowest, which should not be possible to affect with this change
  • rename the option to typing_extensions with a default value of true

};
let (no_return_edit, binding) = checker
.import_from_typing(member, at, PythonVersion::lowest())
.typing_importer(member, PythonVersion::lowest())?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it okay that we treat "typing-extensions" not being available the same as the import failing (will this suppress the diagnostic or only fail to generate the fix?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe all of the calls in this file are okay because they use PythonVersion::lowest. I think we could even expect or unwrap instead of the first ?, if we wanted, because we should always get a TypingImporter for any supported Python version.

ntBre added 3 commits April 28, 2025 13:49
These were only very nearly duplicates before, but after passing in the
`importer` they would have become exact duplicates.
@ntBre
Copy link
Contributor Author

ntBre commented Apr 28, 2025

I went through all of the changed rules and made changes similar to your suggestions above to restructure the code to call typing_importer first, return early if None, and only then to call import. This reverted many of the other code changes I had to make in the first draft.

It also made two functions that were only very similar before now identical: duplicate_union_member::generate_union_fix and redundant_numerical_union::generate_union_fix, so I factored these out into the parent module.

The only change we might not is in the very last commit, where I used anyhow::Context to convert one of the PythonVersion::lowest cases directly into an Err. As mentioned above, I think this is safe because this import should always be available in typing for any Python version we support. We could even unwrap if we wanted, but this is a bit safer just in case.

Thanks for catching these additional benefits of the new API!

@ntBre ntBre merged commit 01a31c0 into main Apr 28, 2025
33 checks passed
@ntBre ntBre deleted the brent/typing-extensions-config branch April 28, 2025 18:57
dcreager added a commit that referenced this pull request Apr 28, 2025
* main: (37 commits)
  [red-knot] Revert blanket `clippy::too_many_arguments` allow (#17688)
  Add config option to disable `typing_extensions` imports  (#17611)
  ruff_db: render file paths in diagnostics as relative paths if possible
  Bump mypy_primer pin (#17685)
  red_knot_python_semantic: improve `not-iterable` diagnostic
  [red-knot] Allow all callables to be assignable to @Todo-signatures (#17680)
  [`refurb`] Mark fix as safe for `readlines-in-for` (`FURB129`) (#17644)
  Collect preview lint behaviors in separate module (#17646)
  Upgrade Salsa to a more recent commit (#17678)
  [red-knot] TypedDict: No errors for introspection dunder attributes (#17677)
  [`flake8-pyi`] Ensure `Literal[None,] | Literal[None,]` is not autofixed to `None | None` (`PYI061`) (#17659)
  [red-knot] No errors for definitions of `TypedDict`s (#17674)
  Update actions/download-artifact digest to d3f86a1 (#17664)
  [red-knot] Use 101 exit code when there's at least one diagnostic with severity 'fatal' (#17640)
  [`pycodestyle`] Fix duplicated diagnostic in `E712` (#17651)
  [airflow] fix typos `AIR312` (#17673)
  [red-knot] Don't ignore hidden files by default (#17655)
  Update pre-commit hook astral-sh/ruff-pre-commit to v0.11.7 (#17670)
  Update docker/build-push-action digest to 14487ce (#17665)
  Update taiki-e/install-action digest to ab3728c (#17666)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

configuration Related to settings and configuration rule Implementing or modifying a lint rule

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PYI019 causes false positives on .py files if you support Python <=3.10

3 participants