Skip to content

Conversation

@k4r1
Copy link
Contributor

@k4r1 k4r1 commented Oct 25, 2019

Issue: #366

Description:
This is a sketch for now, we may want to look at a more sophisticated fix. Although it does fix the linked issue, it does so in a somewhat scorched-earth manner.

Essentially what is happening is that if you have a module which is not directly imported by the base module (e.g. app.py imports a.py, a.py imports b.py), then changes to that module will not be reloaded when a file change is detected.

As far as I can understand, this is what is happening:

  • User changes b.py
  • Reload logic:
    • Deletes b from sys.modules
    • Calls eval on the code from app.py
    • The code in app.py imports a, which is already present in sys.modules, and so we are done
    • The result is that we keep using the old copy of b in memory until a is reloaded

The ideal solution would be to unload both the modified module, as well as any modules that are importing it, but determining this tree of all possible paths for importing a given module may not be worth the effort.

Instead, what this PR does is simply unloads all watched modules. So in the scenario above, a gets unloaded as well when b changes, and so at the eval point both modules get reloaded, therefore reflecting the changes to b in the running app.


Contribution License Agreement

By submiting this pull request you agree that all contributions to this project are made under the Apache 2.0 license.

@k4r1 k4r1 requested a review from a team as a code owner October 25, 2019 09:45
@k4r1 k4r1 force-pushed the fix/reload-depth branch 2 times, most recently from d4275af to 64a79fa Compare October 25, 2019 10:32
@k4r1 k4r1 force-pushed the fix/reload-depth branch from 64a79fa to 88d316b Compare October 25, 2019 10:33
@MarcSkovMadsen
Copy link

Great stuff

Copy link
Contributor

@tvst tvst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, @k4r1 ! Thanks for the contribution, it's a great work-around.

In the future, though, as requested in our pull request template and contributing guidelines, please check in before starting to code on a bug fix or feature request. In this specific case, @tconkling was already working on this issue, which means some effort has been duplicated in this process.

The code looks good to me, but it's possible I missed something. So I'll leave it to @tconkling to take a look before we move ahead with the merge (likely on Tuesday).

Also: please add a unit test for deep imports in lib/tests/streamlit/watcher/LocalSourcesWatcher_test.py

# modules.
for wm in self._watched_modules.values():
if wm.module_name is not None and wm.module_name in sys.modules:
del sys.modules[wm.module_name]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a really slick solution!

Since @tconkling was already looking at this issue, I'll let him take a look before we merge it in.

@tvst tvst requested a review from tconkling October 27, 2019 05:32
@k4r1 k4r1 requested a review from tvst October 29, 2019 13:38
Copy link
Contributor

@tconkling tconkling left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great. It solves #358 as well (which was marked as a duplicate of #366, but I think it's not exactly the same thing)!

Thanks a lot, @k4r1!

@k4r1
Copy link
Contributor Author

k4r1 commented Oct 31, 2019

Hey @tconkling @tvst what's your release cycle like? Could we get a rough estimate for when this might make its way into a release?

tconkling added a commit to tconkling/streamlit that referenced this pull request Nov 4, 2019
* develop:
  Fix cypress tests (streamlit#559)
  Extends capability to send config keys by param to "streamlit hello" command (streamlit#527)
  Explicitly use bash in Makefile (streamlit#556)
  Fix trailing whitespace (streamlit#562)
  Adding missing key parameter to text_input (streamlit#571)
  Create CONTRIBUTING.md
  Don't resize vega chart on update (streamlit#497)
  LaTeX support (streamlit#491)
  Workaround for deep import reloads not working (streamlit#537)
  Add dist-packages and site-packages to blacklist just in case. (streamlit#544)
@k4r1 k4r1 deleted the fix/reload-depth branch November 7, 2019 06:00
@MarcSkovMadsen
Copy link

Like @k4r1 I would really like to have this released as this would simplify my workflow at work and with awesome-streamlit.org.

When can this be expected to be released?

Thanks

@tconkling @tvst

@tvst
Copy link
Contributor

tvst commented Nov 20, 2019

Sorry, I somehow missed this!

This fix was released in version 0.50.x around 9 days ago. LMK if you're still seeing the issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants