Skip to content
This repository was archived by the owner on May 6, 2026. It is now read-only.
This repository was archived by the owner on May 6, 2026. It is now read-only.

After retry limit reached, ResumableUpload.transmit_next_chunk propagates exception with invalid==False #324

@AshleyT3

Description

@AshleyT3

The code below with comments probably best describes the issue... it shows two workarounds. A third workaround (not shown) might be to look for a retry timeout exception and assume the ResumableUpload is invalid despite it being in a recoverable state in my various tests. I'm wondering if a call to _make_invalid upon final retry limit reached is reasonable, or perhaps documenting implied invalidity after retry limit though the latter would seem odd since a retry limit having been reached is not equivalent to a non-recoverable state.

upload = ResumableUpload(upload_url, chunk_size)
response = upload.initiate(transport=session, 
                            stream=stream, 
                            metadata=metadata, 
                            content_type="application/octet-stream", 
                            stream_final=False)
while not upload.finished:
    try:
        pos_before_xmit = stream.tell()
        upload.transmit_next_chunk(session)
    except Exception as ex:
        bytes_attempted = stream.tell() - pos_before_xmit
        """ 
        #
        # I first tried the following which skips recover
        # and jumps back to transmit_next_chunk above.
        # Without this, transmit_next_chunk fails. This
        # seems somewhat equivalent to continuing
        # above with retry reset.
        #
        if not upload.invalid:
            stream.seek(pos_before_xmit, io.SEEK_SET)
        else:
            r = upload.recover(session)
        """
        #
        # After looking at internals, I tried this which
        # also worked. Without the _make_invalid call,
        # recover fails and the stream is not rewound,
        # causing transmit_next_chunk to fail. 
        #
        if not upload.invalid:
            upload._make_invalid() # not really supposed to be called by users.
        r = upload.recover(session)
        # etc.

An aside, thanks to authors/contribs who created ResumableUpload.

Metadata

Metadata

Assignees

Labels

api: storageIssues related to the googleapis/google-resumable-media-python API.priority: p3Desirable enhancement or fix. May not be included in next release.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions