Skip to content

A solution to build Apple Silicon (M1) Python binary wheels #29262

@pietrodn

Description

@pietrodn

Currently, the grpcio Python package does not distribute functioning binary wheels for the Apple Silicon (M1 processor family) architecture.

This has been a major blocker for me and other colleagues working with Google Cloud, as grpcio is a fundamental building block of many Google Cloud client libraries, and compiling grpcio from source on Macs is far from straightforward, as the comments on this issue prove.

Besides, a broken Mac "universal" wheel was uploaded on PyPI, preventing Apple Silicon users on Python 3.10 to even install grpcio from source without passing specific configuration options to pip. Those using poetry are out of luck, as there is no way to force a source installation.

I understand there are significant technical difficulties in providing a working binary wheel for Apple Silicon.
Here is a simple solution to get working binary wheels for Apple Silicon, on Python 3.8, 3.9, 3.10.

The idea is to use a commercially available CircleCI Mac machine executor (on Intel) and cross-compile grpcio for Apple Silicon using the cibuildwheel tool.
As you see, there's no need for specialised Apple Silicon machines, which most CI providers don't offer.

Here is the stripped-down version of the CircleCI pipeline that we are using internally.
It builds the wheel in parallel for Python 3.8, 3.9, and 3.10. The resulting .whl files are saved into the CircleCI artifacts of the job, but they can also easily be uploaded to PyPI using a tool such as twine.

I'm sharing this in the sincere hope you can provide official working Apple Silicon wheels soon. ☮️

version: 2.1

jobs:
  build-grpcio-apple-silicon-wheel:
    working_directory: ~/grpcio-wheels
    macos:
      xcode: << parameters.xcode_version >>

    parameters:
      python_build_version:
        type: string
        description: The Python version to build the wheel for
      xcode_version:
        type: string
        description: The Xcode version for the CircleCI executor
        default: 13.3.0

    environment:
      CIBW_ARCHS_MACOS: "arm64"
      CIBW_TEST_SKIP: "*_arm64"
      CIBW_BUILD: "<< parameters.python_build_version >>-macosx_arm64"
      CIBW_BUILD_FRONTEND: pip
      CIBW_ENVIRONMENT_MACOS: "GRPC_PYTHON_BUILD_WITH_CYTHON=1"
      CIBW_BEFORE_BUILD: "pip install -r requirements.txt"
    steps:
      - run:
          name: Install cibuildwheel
          command: |
              python3 -m venv venv
              source venv/bin/activate
              python -m pip install -U pip wheel cibuildwheel==2.3.1
      - run:
          name: Build grpcio
          command: |
            source venv/bin/activate
            pip download --no-binary grpcio grpcio
            tar xf *.tar.gz
            cd grpcio-*/
            
            cibuildwheel --output-dir ../wheelhouse
      - store_artifacts:
          path: wheelhouse/

workflows:
  version: 2
  build:
    jobs:
      - build-grpcio-apple-silicon-wheel:
          matrix:
            parameters:
              python_build_version: [ "cp38", "cp39", "cp310" ]

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions