Skip to content

sdist race condition and temp dir customization #1222

@rainwoodman

Description

@rainwoodman

Currently if two sdist commands are ran at the same time there is a race condition. One sdist command can be purging the temp files under {packagename}-{packageversion} in the setup.py directory while the other is zipping it, causing error messages like this

Creating tar archive
removing 'cosmo4d-0.0.0' (and everything under it)
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/setup.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/tests/test_engine.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/tests/test_pmesh.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/tests/test_map2d.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/tests'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/iotools.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/pmeshengine.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/__init__.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/version.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/options.py'
Processing /tmp/VlnJRkP/cosmo4d-0.0.0.tar.gz
Skipping bdist_wheel for cosmo4d, due to binaries being disabled for it.

One possible solution to avoid the racing is to allow an argument --temproot to allow redirection of the temp directory to a different location. Then the caller to setup.py (or sdist itself) can use mktemp -d to avoid building sdist from the same temp dir; or at least to minimize the chances of racing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions