Skip to content

upload_blob gets incorrect stream length if passed a socket (get_length returns st_size which is invalid) #19724

@jessewmc

Description

@jessewmc
  • Package Name: azure-storage-blob
  • Package Version: 12.8.1
  • Operating System: debian 10
  • Python Version: 3.7.3

Describe the bug
When passing a socket (e.g. requests raw response) to BlobClient.upload_blob(), upload_blob incorrectly infers a 0 length and does not read any data out of the socket to upload.

The problem appears to be in request_handlers.py def get_length(data). The function calls fstat(fileno).st_size in a try block, but st_size does not seem to have a valid meaning for socket fds, see https://linux.die.net/man/2/fstat. In my environment this returns zero but does not throw an OSError, so get_length erroneously returns 0.

To Reproduce
This minimal code will upload 0 bytes:

import requests
import os
from azure.storage.blob import BlobClient

blob_client = BlobClient.from_connection_string(
    conn_str=os.environ["BLOB_CONN_STR"],
    container_name="blobtest",
    blob_name="test.iso")

with requests.get('https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.10.0-amd64-netinst.iso', stream=True) as resp:
    blob_client.upload_blob(resp.raw, overwrite=True)

Expected behavior
The above code should upload the full iso in chunks.

Additional context
I believe the correct way to handle this is to check if the fd is a regular file or symlink before returning st_size, and if it is not a regular file fall through the rest of the get_length call which ends up returning None which results in the correct behaviour.

I have a pull request I will attach here shortly.

Metadata

Metadata

Assignees

Labels

ClientThis issue points to a problem in the data-plane of the library.Service AttentionWorkflow: This issue is responsible by Azure service team.StorageStorage Service (Queues, Blobs, Files)customer-reportedIssues that are reported by GitHub users external to the Azure organization.needs-team-attentionWorkflow: This issue needs attention from Azure service team or SDK teamquestionThe issue doesn't require a change to the product in order to be resolved. Most issues start as that

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions