New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove deep-freezing of code objects and modules. #108716
Comments
|
Out of interest and unrelated to the other arguments you brought up: Why do you believe that removing deep-freeze for code objects slows down startup time ? If I look at https://github.com/faster-cpython/benchmarking-public/blob/main/results/bm-20230827-3.13.0a0-1ad9bed/bm-20230827-linux-x86_64-faster%252dcpython-no_deep_freeze_3-3.13.0a0-1ad9bed-vs-3.11.0.md which measures removal of deep-freezing code objects against 3.11 which introduced this, the startup time has gone up by 18%. The other differences and overall speedup shown in that benchmark are most likely not related to the deep-freeze change. Background: I'm looking into using deep freeze for eGenix PyRun, which freezes (not deep freezes) most of the stdlib. I haven't done experiments yet (still trying to get things ported to 3.11, which is proving harder than usual due to the many CPython internal changes), so it'd be good to know whether it's worth starting into that adventure. Thanks. |
|
The main advantage of freezing is having to avoid I/O finding and reading .pyc files. We are looking to deep freeze more objects, though. Strings, and tuples of strings are the obvious candidates. |
|
Is there more work to be done, or can we close this? |
|
There is more to be done with deep freezing in general, but this specific task is done. |
Actually, no 🙂 That's the main point of freezing modules using the freeze tool: you avoid I/O and reading PYC files by placing the byte code into a C array. The purpose and intent of deep-freeze was to additionally avoid having to create the code objects from the byte code and put the ready to use objects into memory (again as a C array). So you skip the "read the byte code step" to create the objects and go straight to the objects. Well, at least that's what I had in mind when I discussed this with Guido some time ago.
Ok, mutable objects are probably not well stored in static C arrays 😉
I'd add bytes to that list, since the byte code in code objects is stored as a bytes object, AFAIK. In any case, if this logic will change again for 3.13, it's probably better to wait some more before experimenting with deep-freezing the stdlib as I had planned for eGenix PyRun in one of the next releases. Thanks. |
Remove more references to now unused Python/deepfreeze/deepfreeze.c.
Remove more references to now unused Python/deepfreeze/deepfreeze.c.
…110078) Remove more references to now unused Python/deepfreeze/deepfreeze.c.
Deep-freezing, as implemented has a number of problems:
makewill not recreate deep-frozen modulespycfiles.So let's remove it.
But let's keep the ability to deep-freeze objects.
The ability to deep-freeze objects into the executable is valuable, we already do it with strings, and deep-freezing whole immutable object graphs like the
sysandbuiltinmodules could improve start up considerably.Linked PRs
The text was updated successfully, but these errors were encountered: