Switch to a Python/YAML based build system for wheels#27
Switch to a Python/YAML based build system for wheels#27bgruening merged 16 commits intogalaxyproject:masterfrom
Conversation
thing a bit cleaner.
Use setarch/linux32 to fool Python into thinking the system is 32 bit (rather than modifying the source). Improvements to the YAML-based wheel build system to (hopefully) allow building on RHEL-based distros.
1. Base OS images (32- and 64-bit) for Debian 6-9, Ubuntu 12.04 and 14.04, and CentOS 6 and 7 (RHEL 7 is 64-bit only). 2. Wheel build images (Python 2.6 and 2.7, UCS2 and UCS4) on all those base OS images. Wheel build images include the evolving modification to the Python wheel package that I'm working on. What this will now do: - Build "generic" wheels that'll run on platforms at least as old as Debian Squeeze (so everything listed above). - Build "targeted" wheels of things with complex/important system dependencies. The case for this is psycopg2, which will use the system libpq and have SSL support, neither of which we would then be responsible for keeping up to date.
|
The discussion on Distutils-SIG continues so this may need to change later, but this should hopefully be enough to get us out the door for now. I'd like to add openSUSE images (which I assume would build wheels compatible with SLES) since SLES is a moderately popular distro in Europe. I'll try to get the images uploaded to docker hub at some point, but I want to take a moment to figure out why the wheel images seem to be so big (natefoo/centos-7 is only 650 MB whereas natefoo/centos7-wheel is 1.6GB). |
wheels/image/Makefile
Outdated
There was a problem hiding this comment.
This seems a bit off to me, but I don't have a better solution. I tried to write a response using makefile variables (e.g. make -f Makefile.template DIST=debian BITS=64 ...) but that wasn't any better than this.
There was a problem hiding this comment.
Funny, this is a replacement of what I'd originally wrote, which used Makefile variables...
I was originally trying to have make generate the Makefiles, but that proved to be pretty difficult to get make to cooperate with. But I am no make expert.
they've been built, don't continue if they fail to build.
build.py) so that it can use the improved platform detection.
1. Standardize tagging scheme to mirror the official images. 2. Switch to the "official" base images for the OSes. Originally I wanted to have 32- and 64- bit base images be as close to identical as possible, but I think that's kind of pointless, and using official images is nice for both myself and anyone else who's going to build wheels. 2. Use Debian multiarch support to avoid having to build 32-bit base images on Debian >= 8, Ubuntu >= 14.04.
install openssl-devel for reasons that I don't really want to figure out because honestly who is using Fedora 20? Nobody, that's who.
|
Now with 100% less WIP tag! |
|
LGTM/+1, but untested. Glad to see wheels are finally happening! |
|
@erasche The images are currently uploading to Docker Hub in case you actually wanted to test the wheel building. If you wanted to test the image building, glwt. If you didn't want to test anything I'll just click the 'ole merge button there after I've done a few more wheel builds of my own. |
|
@natefoo go for it |
Switch to a Python/YAML based build system for wheels
Many things accomplished in this PR:
natefoo/base-wheelandnatefoo/base32-wheelwill be used to build generic images.CentOS 7 does not have a 32-bit image because it is not present in RHEL/CentOS 7. I kept 32-bit support for current versions of Debian and Ubuntu just because I figure them (or derivatives) the most likely for being installed on laptops, which is probably the only likely case where you'll find 32-bit Linux anymore. Nothing else has 32-bit support other than the generic wheels.