Skip to content

🚨 Bump to Python 3.10 and rework how we check 3rd-party libraries existence#41268

Merged
Cyrilvallez merged 16 commits into
mainfrom
better-check-package
Oct 6, 2025
Merged

🚨 Bump to Python 3.10 and rework how we check 3rd-party libraries existence#41268
Cyrilvallez merged 16 commits into
mainfrom
better-check-package

Conversation

@Cyrilvallez

@Cyrilvallez Cyrilvallez commented Oct 1, 2025

Copy link
Copy Markdown
Member

What does this PR do?

Python 3.9 is officially EOL starting this month, so let's bump it! Took the opportunity of the new features available in 3.10 to simplify quite a bit our library checks

Note that it is breaking BC a bit as I removed some old and unused library checks that were public members

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ydshieh ydshieh left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice.

Let's wait python 3.9 reaches EOL and merge.

(or if v5 is released before that date, we can also drop the support of 3.9 at the same day)

@Cyrilvallez

Copy link
Copy Markdown
Member Author

Yeah, the exact date was not super clear to me @ydshieh 🥲, do you have more infos from experience? Official website only says October, without a given day, so I guess it's already EOL? Or at the end of the month? (but very unlikely they would have a last release in the last month anyway I guess)

@Cyrilvallez Cyrilvallez changed the title Bump to Python 3.10 and associated improvements Bump to Python 3.10 and rework how we check 3rd-party libraries existence Oct 2, 2025
@ydshieh

ydshieh commented Oct 2, 2025

Copy link
Copy Markdown
Collaborator

do you have more infos from experience?

No. From previous python versions, it is definitely not something predictable (i.e. not always the end of the month):

3.8 2024-10-07
3.7 2023-06-27
3.6 2021-12-23
3.5 2020-09-30

@Cyrilvallez Cyrilvallez changed the title Bump to Python 3.10 and rework how we check 3rd-party libraries existence 🚨 Bump to Python 3.10 and rework how we check 3rd-party libraries existence Oct 2, 2025

@ArthurZucker ArthurZucker left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure lru cache is required no?

Comment thread src/transformers/utils/import_utils.py
@Cyrilvallez

Cyrilvallez commented Oct 2, 2025

Copy link
Copy Markdown
Member Author

Not strictly required, but as the output of those functions cannot change anyway, it's an easy and nice way to cache the result without having to define 10k global variables as was done previously. About one third/half of them had it previously already
It's basically a tiny performance optimization

@Cyrilvallez

Copy link
Copy Markdown
Member Author

Merging as we released last v4 version already!

@Cyrilvallez Cyrilvallez merged commit 55b172b into main Oct 6, 2025
26 checks passed
@Cyrilvallez Cyrilvallez deleted the better-check-package branch October 6, 2025 09:04
@lhoestq

lhoestq commented Oct 6, 2025

Copy link
Copy Markdown
Member

(note that transformers@main can still be installed on 3.9 since it's not completely dropped in setup.py and raises AttributeError: module 'importlib.metadata' has no attribute 'packages_distributions')

@Cyrilvallez

Copy link
Copy Markdown
Member Author

Indeed, I forgot it! It was fixed in #41410 already, thanks for checking-in!

@gante gante mentioned this pull request Oct 11, 2025
5 tasks
AhnJoonSung pushed a commit to AhnJoonSung/transformers that referenced this pull request Oct 12, 2025
…stence (huggingface#41268)

* cleanup

* add check

* fix

* remove all global variables

* fix

* add lru caches everywhere

* fix

* fix

* style

* improve

* reorder all functions

* fix order

* improve

* fix

* fix

* fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants