Skip to content

Add LDFLAGS for zlib on macOS >= 1100#1711

Merged
joshfriend merged 1 commit intopyenv:masterfrom
gensym-x:issue1643
Nov 14, 2020
Merged

Add LDFLAGS for zlib on macOS >= 1100#1711
joshfriend merged 1 commit intopyenv:masterfrom
gensym-x:issue1643

Conversation

@gensym-x
Copy link
Copy Markdown
Contributor

@gensym-x gensym-x commented Oct 7, 2020

Add LDFLAGS for Xcode SDK zlib on macOS >= 1100 to resolve build issues on Big Sur.

Make sure you have checked all steps below.

Prerequisite

  • Please consider implementing the feature as a hook script or plugin as a first step.
    • pyenv has some powerful support for plugins and hook scripts. Please refer to Authoring plugins for details and try to implement it as a plugin if possible.
      n/a
  • Please consider contributing the patch upstream to rbenv, since we have borrowed most of the code from that project.
    • We occasionally import the changes from rbenv. In general, you can expect changes made in rbenv will be imported to pyenv too, eventually.
    • Generally speaking, we prefer not to make changes in the core in order to keep compatibility with rbenv.
      n/a - this code does not appear in rbenv
  • My PR addresses the following pyenv issue (if any)

Description

  • Here are some details about my PR
    This is a 3 line change to python-build that affects macOS 11.0 Big Sur (currently in beta). Big Sur no longer packages dynamic libraries in /usr/lib as detailed in Unable to build Python on macOS Big Sur with Xcode 12 beta #1643 (comment) . This requires python-build to link against $(xcrun --show-sdk-path)/usr/lib on macOS >= 11.0 when using the macOS-shipped zlib. This allows python-build to correctly build python again on macOS 11 (Big Sur).

Tests

  • My PR adds the following unit tests (if any)
    n/a

Add LDFLAGS for Xcode SDK zlib on macOS >= 1100 to resolve build issues on Big Sur.
@archoversight
Copy link
Copy Markdown

Now that Big Sur is available for realz, it would be great to have this in pyenv :-)

@joshfriend joshfriend merged commit 943015e into pyenv:master Nov 14, 2020
@armenzg
Copy link
Copy Markdown

armenzg commented Nov 24, 2020

Hello all,
Anyone know when the next release of pyenv will happen?
Thank you so much for your hard work!

armenzg added a commit to getsentry/sentry that referenced this pull request Dec 9, 2020
Installing Python via pyenv on Big Sur for older Python versions requires:
* Setting LDFLAGS for zlib (pyenv/pyenv#1711)
* Patching the source code

Once a new release of pyenv is released and we use a newer Python 3 version
we will not need all these hacks and all Mac versions will be able to use
the same code paths.

Fixes #22497
@tillhainbach
Copy link
Copy Markdown
Contributor

@joshfriend This PR may actually cause the build to fail compiling readline from home-brew. At least on both of my macs running on macOS Big Sur 11.0.1. It will print

WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?

The error could not have been caught by travis-ci because the pipeline does not include macOS 11 (yet).
I commented-out the added lines and exported the LDFLAG manually with:

export LDFLAGS='-L /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/'

This time, pyenv installed/compiled python with no errors or warnings.

So, it turns out that ${xc_sdk_path} was using the MacOSX11.0.sdk from Xcode.app (Version 12.2 (12B45b)) and the sdk in /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/ was still 10.15.
Can someone confirm similar issues?

@tillhainbach
Copy link
Copy Markdown
Contributor

So I did some more digging and apparently the order in LDFLAGS is somehow relevant (?)

    if is_mac -ge 1100; then
      export LDFLAGS="-L${xc_sdk_path}/usr/lib ${LDFLAGS}"
    fi

--> Warning ...

    if is_mac -ge 1100; then
      export LDFLAGS="${LDFLAGS} -L${xc_sdk_path}/usr/lib"
    fi

--> no warning

@tillhainbach
Copy link
Copy Markdown
Contributor

Edit: I updated the travis setup and the python-build-script to force travis to build against macOS SDK 11.0 (osx_image xcode12.2) and was able to reproduce the warning that readline extension was not compiled. However, this is not treated as a build error and hence the checks passed. But you can see the warning in the travis-logs. See example here: https://github.com/tillhainbach/pyenv/pull/8 and logs here https://travis-ci.com/github/tillhainbach/pyenv/jobs/457193096#L9265

So, it should be considered merging #1754

armenzg added a commit to getsentry/sentry that referenced this pull request Dec 10, 2020
Installing Python via pyenv on Big Sur for older Python versions requires:
* Setting LDFLAGS for zlib (pyenv/pyenv#1711)
* Patching the source code

Once a new release of pyenv is released and we use a newer Python 3 version
we will not need all these hacks and all Mac versions will be able to use
the same code paths.

Fixes #22497
armenzg added a commit to getsentry/sentry that referenced this pull request Dec 11, 2020
Installing Python via pyenv on Big Sur for older Python versions requires:
* Setting LDFLAGS for zlib (pyenv/pyenv#1711)
* Patching the source code

Once a new release of pyenv is released and we use a newer Python 3 version
we will not need all these hacks and all Mac versions will be able to use
the same code paths.

Fixes #22497
@hjmallon
Copy link
Copy Markdown

@tillhainbach did you consider any other methods than adding a linker flag? It seems homebrew are using -isysroot for this stuff instead, I'n not sure whether that should make much difference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants