Skip to content

Executable script wrappers fail in cross-build environments #230

@freakboy3742

Description

@freakboy3742

Describe the bug

When you install a tool with an entry point, pip creates a script in the bin directory of your environment. This capability is based on a vendored copy of distutils.

ScriptMaker._build_shebang() interrogates the path of the Python interpreter that will be used to start the script; if that path is more than 127 characters long (512 on macOS), or contains a space, an exec wrapper is used.

However - in crossenv-based build environments, the "Python binary" isn't a literal binary - it is a shell script that invokes the Python binary after performing some environment modifications. This is needed because cross-environment builds can't literally run python for the host platform (e.g., you can't run an iOS or Android binary on machine being used to run the build)

If Python code attempts to os.execv() an entry point script that is in the "simple" form, it will raise OSError [Errno 8] Exec format error, because the shebang isn't a literal binary. The "non-simple" form works fine, because the "actual shebang" is /bin/sh. This means that the exact directory in which you create your crossenv virtual environment (and thus the wrapped host python binary path) will alter how distutils works.

To Reproduce

This is difficult to reproduce, as it requires a cross build environment. It was observed in the context of beeware/mobile-forge#60; however, the branch of crossenv has been patched to avoid the problem.

If you checkout mobile-forge into /tmp, then set up a mobile-forge environment as described in the README, downgrade crossenv to 8fbcf5f, then build numpy with forge iphoneos:arm64 numpy, you'll get the error. This requires a macOS machine with Xcode.

Expected behavior

The entry point script should always be executable with os.exec()

Environment

  • OS, including version: macOS 14.6
  • Version of this library: 0.3.8 (as vendored into pip 24.2)

Additional information

If a crossenv specific fix is acceptable, the fix is to look for sys.cross_compiling as an additional check in ScriptMaker._build_shebang. This isn't a standardised value, but it's not an unreasonable name or location for the functionality.

Alternatively, an additional sys.platform check could be used to inspect for platforms that are known to be cross compiling - ios and android would be the obvious candidates, but there could be others.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions