[Python] pyarrow.fs.copy_files doesn't work if filenames contain special characters #33468
Description
I can't upload a file called spam=ham to a filesystem that emulates an S3 API. I can workaround the issue by renaming the file spam-ham.
To reproduce, run a filesystem that emulates an S3 API:
docker run -p 9444:9000 scireum/s3-ninja:latestAuthenticate with the filesystem:
export AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"Then run this Python script:
import os
import tempfile
import pyarrow.fs
source = tempfile.mkdtemp()
file_path = os.path.join(source, "spam=ham")
open(file_path, "w").close()
filesystem, path = pyarrow.fs.FileSystem.from_uri(
"s3://bucket?scheme=http&endpoint_override=localhost:9444"
)
pyarrow.fs.copy_files(source, path, destination_filesystem=filesystem)You'll get the error
OSError: When initiating multiple part upload for key 'spam=ham' in bucket 'bucket': AWS Error [code 22]: The computed request signature does not match the one provided. Check login credentials. (Expected: e70ab9efb620f744abd43d13e8e6846c585a41da543bfb5da67d2fe1ccfd1aaa, Found: 648456e3441dad5a014b2981c71b6e69ccac9732bdcdbe2d363d95105d914340)This issue is motivated by ray-project/ray#29845.
Reporter: Balaji Veeramani
Assignee: Antoine Pitrou / @pitrou
PRs and other links:
Note: This issue was originally created as ARROW-18290. Please see the migration documentation for further details.