Fix _LazyModule to not trigger load for introspection attributes by kmcgrady · Pull Request #21620 · pulumi/pulumi

kmcgrady · 2026-01-29T19:06:07Z

Summary

Adds early return in _LazyModule.__getattribute__ to skip lazy loading for standard module introspection attributes (__file__, __spec__, __path__, __name__, __loader__, __package__, __cached__, __doc__, __dict__, __class__)
Prevents tools like debuggers, file watchers, and IDEs from triggering full module initialization when inspecting sys.modules
Fixes ~340MB memory overhead and ~3 second delays when tools like Streamlit iterate over lazy-loaded Pulumi submodules. Specifically Streamlit looks at module information in tracking changes in files and other use cases. This caused unnecessary lazy loading.

Test plan

Added test_lazy_module_introspection_attrs_do_not_trigger_load test that verifies:
- Accessing introspection attributes keeps module in lazy state
- Accessing real attributes still triggers loading as expected
All existing test_utils.py tests pass

🤖 Generated with Claude Code

Tools like debuggers, file watchers, and IDEs (e.g., Streamlit) iterate over sys.modules and check module attributes like __file__, __spec__, __path__, etc. The previous implementation triggered a full module load on any attribute access, causing ~340MB memory overhead and ~3 second delays when these tools inspected lazy-loaded submodules. This fix adds an early return in __getattribute__ that checks if the requested attribute is a standard introspection attribute and returns it directly from the parent class without triggering the lazy load. See: streamlit/streamlit#13530 Co-Authored-By: Claude (claude-opus-4-5) <noreply@anthropic.com>

github-actions · 2026-01-29T19:06:24Z

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

/run-acceptance-tests - used to test run the acceptance tests for the project
/run-codegen - used to test the Pull Request against downstream codegen
/run-docs-gen - used to test the Pull Request against documentation generation

Co-Authored-By: Claude (claude-opus-4-5) <noreply@anthropic.com>

github-actions · 2026-01-29T19:20:26Z

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

/run-acceptance-tests - used to test run the acceptance tests for the project
/run-codegen - used to test the Pull Request against downstream codegen
/run-docs-gen - used to test the Pull Request against documentation generation

kmcgrady · 2026-02-03T17:58:03Z

Hey @Frassle . Wondering if we can get this change reviewed (or if there are any processes to get this PR noticed)? It seems small and innocent enough, and it would benefit Streamlit developers who use Pulumi.

tgummerer

Just a small comment from me, I don't know enough about this code to review it for correctness. This could use some more description for why it is correct to shortcircuit for these attributes as well.

tgummerer · 2026-02-04T09:27:07Z

+        if mod_name.startswith("lazy_import_test"):
+            del sys.modules[mod_name]
+
+    # Create a lazy module


All of these comments below are not very useful. AI likes to add exactly what it does as a comment for whatever reason, but just remove them please, they are just noise.

Ack. Happy to remove.

Frassle

I'm not sure this is correct? Doesn't this mean any lookup of like __file__ or __doc__ returns the information from types.ModuleType rather than the module itself?

kmcgrady · 2026-02-04T14:47:53Z

I'm not sure this is correct? Doesn't this mean any lookup of like file or doc returns the information from types.ModuleType rather than the module itself?

This is using the __get_attribute__ from the parent class (types.ModuleType) but on the LazyModule itself (this bypasses the infinite recursive call to the same method. The fields should be set https://github.com/kmcgrady/pulumi/blob/fix/lazy-load-module/sdk/python/lib/pulumi/_utils.py#L114, so it should be providing the correct spec details.

I saw that @justinvp has done some work in this space and can probably verify.

Frassle · 2026-02-04T16:55:51Z

so it should be providing the correct spec details.

Can we update the test to show that then? Once that's done and it works then I think we can merge this.

- Expand comment explaining why short-circuiting is correct: the introspection attributes are set on the module instance by importlib.util.module_from_spec() before exec_module() is called - Remove inline comments from test per reviewer request - Add assertions verifying introspection attrs return correct values (__name__, __spec__.name, __file__, __package__) Co-Authored-By: Claude (claude-opus-4-5) <noreply@anthropic.com>

github-actions · 2026-02-04T19:53:58Z

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

/run-acceptance-tests - used to test run the acceptance tests for the project
/run-codegen - used to test the Pull Request against downstream codegen
/run-docs-gen - used to test the Pull Request against documentation generation

Frassle · 2026-02-04T20:40:07Z

/run-acceptance-tests
Please view the results of the acceptance tests Here

github-actions · 2026-02-05T00:56:14Z

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

/run-acceptance-tests - used to test run the acceptance tests for the project
/run-codegen - used to test the Pull Request against downstream codegen
/run-docs-gen - used to test the Pull Request against documentation generation

Frassle · 2026-02-05T07:30:46Z

/run-acceptance-tests
Please view the results of the acceptance tests Here

kmcgrady · 2026-02-05T16:09:57Z

@Frassle Looks like the tests failed on something with dotnet, so I am not sure if it's related to my change. Let me know if I should rebase or do something else.

Frassle · 2026-02-05T16:14:44Z

No just a flake we've been hitting from too much parallelism in the test runner. I think this should be ok to merge now.

Frassle · 2026-02-05T18:36:43Z

Thanks for the contribution @kmcgrady

To be merged after #21727 and #21764 made it in. Tentative changelog: ## (2026-02-10) ### Features - [cli] Show environment variables that were set if a snapshot integrity error happens [#21709](#21709) - [engine] Pass replacement trigger through to Construct [#21408](#21408) - [engine] Add EnvVarMappings resource option for provider resources, allowing environment variables to be remapped before being passed to the provider [#21572](#21572) - [pkg] BREAKING: Deprecate github.com/pulumi/pulumi/pkg/v3/codegen/dotnet in favor of github.com/pulumi/pulumi-dotnet/pulumi-language-dotnet/v3/codegen. This package will be removed from pulumi/pulumi soon! [#21720](#21720) ### Bug Fixes - [cli] Retry `yarn install` when it fails (e.g. during `pulumi install`) [#21707](#21707) - [engine] Deal with errors in elided journal entries correctly [#21576](#21576) - [sdk/python] Fix `_LazyModule` to not trigger full module load for introspection attributes [#21620](#21620) - [sdkgen/python] Remove workaround for slow typechecking with MyPy and PyCharm [#21722](#21722) ### Miscellaneous - [cli] Write logfile location if verbosity is >= 1 to stderr instead of stdout [#21663](#21663)

pulumi-bot · 2026-02-10T17:11:43Z

This PR has been shipped in release v3.220.0.

kmcgrady requested a review from a team as a code owner January 29, 2026 19:06

Add changelog entry for _LazyModule introspection fix

488dd23

Co-Authored-By: Claude (claude-opus-4-5) <noreply@anthropic.com>

tgummerer reviewed Feb 4, 2026

View reviewed changes

Frassle reviewed Feb 4, 2026

View reviewed changes

Fix formatting

47d9394

Frassle approved these changes Feb 5, 2026

View reviewed changes

Frassle enabled auto-merge February 5, 2026 16:15

Frassle added this pull request to the merge queue Feb 5, 2026

Frassle self-assigned this Feb 5, 2026

Merged via the queue into pulumi:master with commit 369fdb6 Feb 5, 2026
8 checks passed

tgummerer mentioned this pull request Feb 10, 2026

Freeze 3.220.0 #21772

Merged

Conversation

kmcgrady commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

github-actions Bot commented Jan 29, 2026

Uh oh!

github-actions Bot commented Jan 29, 2026

Uh oh!

kmcgrady commented Feb 3, 2026

Uh oh!

tgummerer left a comment

Choose a reason for hiding this comment

Uh oh!

tgummerer Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

kmcgrady Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Frassle left a comment

Choose a reason for hiding this comment

Uh oh!

kmcgrady commented Feb 4, 2026

Uh oh!

Frassle commented Feb 4, 2026

Uh oh!

github-actions Bot commented Feb 4, 2026

Uh oh!

Frassle commented Feb 4, 2026 • edited by pulumi-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Feb 5, 2026

Uh oh!

Frassle commented Feb 5, 2026 • edited by pulumi-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kmcgrady commented Feb 5, 2026

Uh oh!

Frassle commented Feb 5, 2026

Uh oh!

Uh oh!

Frassle commented Feb 5, 2026

Uh oh!

pulumi-bot commented Feb 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kmcgrady commented Jan 29, 2026 •

edited

Loading

Frassle commented Feb 4, 2026 •

edited by pulumi-bot

Loading

Frassle commented Feb 5, 2026 •

edited by pulumi-bot

Loading