Skip to content

Conversation

@pfafflabatuiuc
Copy link
Contributor

@pfafflabatuiuc pfafflabatuiuc commented Nov 21, 2022

When entering the FileOpener context manager, a file lock is now created with the following filename structure " ~<file_name>.lock". All of the functions in the datadict_storage.py module use the FileOpener internally.

All of the functions that use the FileOpener now also have a new argument file_timeout to pass a different amount of time for the timeout of a lock file. This is because in my testing when files start to get big, sometimes a writer or reader takes longer than the timeout time, making the program crash.

I have also created a simple test that tests if the FileOpener creates the lock file, crashes program and checks if the lock file is still being deleted afterwards. I wasn't sure what else add to the unit tests.

As a little convenience bonus, all of the storage functions now accept both str and Path instead of just str and converting them to Path in the function.

@marcosfrenkel marcosfrenkel linked an issue Nov 21, 2022 that may be closed by this pull request
Copy link
Contributor

@wpfff wpfff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good.

Copy link
Contributor

@wpfff wpfff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good.

@wpfff wpfff merged commit 1d04137 into toolsforexperiments:master Nov 22, 2022
@pfafflabatuiuc pfafflabatuiuc deleted the file_locking_fixes branch November 22, 2022 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Have file related objects accept Path like adresses.

2 participants