-
Notifications
You must be signed in to change notification settings - Fork 11.1k
A solution to build Apple Silicon (M1) Python binary wheels #29262
Description
Currently, the grpcio Python package does not distribute functioning binary wheels for the Apple Silicon (M1 processor family) architecture.
This has been a major blocker for me and other colleagues working with Google Cloud, as grpcio is a fundamental building block of many Google Cloud client libraries, and compiling grpcio from source on Macs is far from straightforward, as the comments on this issue prove.
Besides, a broken Mac "universal" wheel was uploaded on PyPI, preventing Apple Silicon users on Python 3.10 to even install grpcio from source without passing specific configuration options to pip. Those using poetry are out of luck, as there is no way to force a source installation.
I understand there are significant technical difficulties in providing a working binary wheel for Apple Silicon.
Here is a simple solution to get working binary wheels for Apple Silicon, on Python 3.8, 3.9, 3.10.
The idea is to use a commercially available CircleCI Mac machine executor (on Intel) and cross-compile grpcio for Apple Silicon using the cibuildwheel tool.
As you see, there's no need for specialised Apple Silicon machines, which most CI providers don't offer.
Here is the stripped-down version of the CircleCI pipeline that we are using internally.
It builds the wheel in parallel for Python 3.8, 3.9, and 3.10. The resulting .whl files are saved into the CircleCI artifacts of the job, but they can also easily be uploaded to PyPI using a tool such as twine.
I'm sharing this in the sincere hope you can provide official working Apple Silicon wheels soon. ☮️
version: 2.1
jobs:
build-grpcio-apple-silicon-wheel:
working_directory: ~/grpcio-wheels
macos:
xcode: << parameters.xcode_version >>
parameters:
python_build_version:
type: string
description: The Python version to build the wheel for
xcode_version:
type: string
description: The Xcode version for the CircleCI executor
default: 13.3.0
environment:
CIBW_ARCHS_MACOS: "arm64"
CIBW_TEST_SKIP: "*_arm64"
CIBW_BUILD: "<< parameters.python_build_version >>-macosx_arm64"
CIBW_BUILD_FRONTEND: pip
CIBW_ENVIRONMENT_MACOS: "GRPC_PYTHON_BUILD_WITH_CYTHON=1"
CIBW_BEFORE_BUILD: "pip install -r requirements.txt"
steps:
- run:
name: Install cibuildwheel
command: |
python3 -m venv venv
source venv/bin/activate
python -m pip install -U pip wheel cibuildwheel==2.3.1
- run:
name: Build grpcio
command: |
source venv/bin/activate
pip download --no-binary grpcio grpcio
tar xf *.tar.gz
cd grpcio-*/
cibuildwheel --output-dir ../wheelhouse
- store_artifacts:
path: wheelhouse/
workflows:
version: 2
build:
jobs:
- build-grpcio-apple-silicon-wheel:
matrix:
parameters:
python_build_version: [ "cp38", "cp39", "cp310" ]