-
Notifications
You must be signed in to change notification settings - Fork 523
Updates Tz Singletons and Caches with WeakValueDictionarys
#672
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@pganssle - I'm trying to figure out how my pytest failed on the AppVeyor CI. It failed on a line that asserts that the weak references are cleared immediately following a couple Any ideas how this could happen? Is it possible that the |
dateutil/utils.py
Outdated
| from datetime import datetime, time | ||
|
|
||
|
|
||
| class FakeWeakRef: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.utils is a public module and very much not the right place for this. It should probably be in tz._common.py
dateutil/tz/_factories.py
Outdated
| from datetime import timedelta | ||
|
|
||
| try: | ||
| import weakref |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would change this to:
try:
from weakref import WeakRefDictionary
except ImportError:
WeakRefDictionary = dictThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually it turns out that Python 2 has weak references, so we can just drop the whole compatibility shim. My bad.
dateutil/test/test_tz.py
Outdated
| from functools import partial | ||
|
|
||
| IS_WIN = sys.platform.startswith('win') | ||
| NOT_PY3 = not sys.version_info[0] == 3 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Normally you would not have to define this. There's already a six.PY3 and six.PY2 that will give you that.
That said, I may have been wrong about weak references being a Python 3 thing. Seems like they are available in Python 2.7, so we should drop this skip test.
|
Hard to tell if this is just holding a strong reference somewhere we don't realize or if the assumptions about one of the inputs is wrong. Seems to only happen on Windows on old versions of Python, though. |
|
Sorry for the flurry of seemingly random commits. I can't seem to replicate AppVeyor's failure on my local machine even after matching the Python version, so I'm trying to troubleshoot with small changes. |
|
Just reverted all the troubleshooting commits, but I'm kind of at a loss here for how this is failing on the AppVeyor CI. The stdout from the 46bdab7 commit is really what's really tripping me up: In this commit, I print the keys inside the I'll keep thinking about how to get around this. One idea I had was to have some kind of |
|
I'm guessing that this is a real issue with Windows and not a problem with the tests. I'll grab this branch and try debugging it on a Windows VM. |
|
Ah, I figured this out. It's because |
|
Cleaned up the commit history. Will merge when tests pass. Thanks so much @cs-cordero! |
|
Happy to help. Thanks for taking it home with that clutch Windows |
|
Hm.. The Python 3.5 test failed but succeeded on reboot. This "id won't get reused" gambit may be less stable than I hoped. I just had a new idea for a more robust test, though. Create a new weak reference to the object before deleting it and make sure its referrent is gone after the gc step. |
|
@cs-cordero One last thing, can you add yourself to |
|
Will do re: |
|
@cs-cordero No this test assumes that the If you do: # Checkout base of the branch
git checkout e6644a4
# Checkout the commit where the tests were added
git checkout 75ae199 -- dateutil/test/test_tz.py
pytest -k test_tz.pyYou'll see that these three tests fail, which means the problem - that the cache itself holds a strong reference to the cached time zones - is solved by this patch. |
|
Ah yes that makes perfect sense! Thank you for explaining. |
This is a more reliable mechanism to test that nothing continues to hold a strong reference to the cached values. I have also switched the tests over to using pytest-style bare functions, and marked them all as smoke tests, since this will not actually be a guaranteed behavior of the interface.
|
@cs-cordero You'll probably need to do a force pull when you edit this branch again, because I've rebased your branch and rewritten the history. If you don't know how to do that it's |
|
🎉 |
Closes #635.