Skip to content

Transfer manager API failure when using process as worker type #1044

@yoland68

Description

@yoland68

Environment details

  • OS type and version: macOS Big Sur 11.2.3
  • Python version: Python 3.10.1
  • pip version: pip 22.0.4
  • google-cloud-storage version: 2.9.0

Steps to reproduce

  1. Call transfer_manager.upload_many_from_files() function with worker type as "process".
  2. This generates an error when it tries to pickle a blob file.

Code example

from google.cloud.storage import transfer_manager
# assuming bucket, client and filepaths have been defined

transfer_manager.upload_many_from_files(bucket, client, filepaths, process=8)

Stack trace:

File "/Users/x/opt/anaconda3/envs/drip-backend/lib/python3.10/site-packages/google/cloud/storage/transfer_manager.py", line 910, in _pickle_blob
p.dump(blob)
TypeError: cannot pickle '_cffi_backend.FFI' object

The file itself is a mp4 video

Metadata

Metadata

Assignees

Labels

api: storageIssues related to the googleapis/python-storage API.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions