Skip to content

[Bug]: Multi-arch Beam SDK Python 3.11 container missing libgeos-dev for ARM #28035

@celeste-zeng

Description

@celeste-zeng

What happened?

Used a multi-arch Beam SDK Python 3.11 container to run a Dataflow pipeline on ARM and the container failed to start due to the error OSError: Could not find lib geos_c or load any of its variants ['libgeos_c.so.1', 'libgeos_c.so'].

Reproduce steps:

  1. Create an ARM VM with docker installed;
  2. Pull and run the multi-arch Beam SDK Python 3.11 container: docker run -it --entrypoint=/bin/bash gcr.io/dataflow-build/siyuez/mock-release/beam_python3.11_sdk:release-test;
  3. Use the python interpreter, run: python;
  4. then run: from apache_beam.io.gcp import bigquery_schema_tools.
    The error will appear: File "/usr/local/lib/python3.11/site-packages/shapely/geos.py", line 54, in load_dll raise OSError( OSError: Could not find lib geos_c or load any of its variants ['libgeos_c.so.1', 'libgeos_c.so'].

On the VM, the error can be fixed by installing the libgeos-dev package: run apt update and thenapt install libgeos-dev. The package takes 4327 kB. We should install it in the Dockerfile for Python 3.11 when the target architecture is ARM.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2bugdone & doneIssue has been reviewed after it was closed for verification, followups, etc.python

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions