Skip to content
This repository was archived by the owner on Mar 31, 2026. It is now read-only.

feat(zb-experimental): implement download_ranges#1551

Merged
chandra-siri merged 74 commits into
mainfrom
bidi_reads_6_read_obj_stream
Sep 29, 2025
Merged

feat(zb-experimental): implement download_ranges#1551
chandra-siri merged 74 commits into
mainfrom
bidi_reads_6_read_obj_stream

Conversation

@chandra-siri

@chandra-siri chandra-siri commented Sep 18, 2025

Copy link
Copy Markdown
Collaborator

Downloads multiple byte ranges from the object into the buffers provided by user.

This interface provides option to download multiple ranges of a GCS Object
concurrently.

Example usage

        client = AsyncGrpcClient().grpc_client
        mrd = await AsyncMultiRangeDownloader.create_mrd(
            client, bucket_name="chandrasiri-rs", object_name="test_open9"
        )
        my_buff1 = open('my_fav_file.txt', 'wb')
        my_buff2 = BytesIO()
        my_buff3 = BytesIO()
        my_buff4 = any_object_which_provides_BytesIO_like_interface()
        results_arr, error_obj = await mrd.download_ranges(
            [
                (0, 100, my_buff1),
                (100, 20, my_buff2),
                (200, 123, my_buff3),
                (300, 789, my_buff4),
            ]
        )
        if error_obj:
            print("Error occurred: ")
            print(error_obj)
            print(
                "please issue call to `download_ranges` with updated"
                "`read_ranges` based on diff of (bytes_requested - bytes_written)"
            )

        for result in results_arr:
            print("downloaded bytes", result)

@product-auto-label product-auto-label Bot added size: xl Pull request size is extra large. api: storage Issues related to the googleapis/python-storage API. labels Sep 18, 2025
@chandra-siri chandra-siri changed the base branch from main to bidi_reads_5_read_obj_stream September 18, 2025 06:39
@product-auto-label product-auto-label Bot added size: m Pull request size is medium. and removed size: xl Pull request size is extra large. labels Sep 18, 2025
Base automatically changed from bidi_reads_5_read_obj_stream to main September 24, 2025 05:28
@chandra-siri chandra-siri marked this pull request as ready for review September 24, 2025 06:33
@chandra-siri chandra-siri requested review from a team September 24, 2025 06:33
@product-auto-label product-auto-label Bot added size: m Pull request size is medium. and removed size: l Pull request size is large. labels Sep 24, 2025
@product-auto-label product-auto-label Bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Sep 24, 2025
Comment thread google/cloud/storage/_experimental/asyncio/async_multi_range_downloader.py Outdated
Comment thread google/cloud/storage/_experimental/asyncio/async_multi_range_downloader.py Outdated
@chandra-siri

Copy link
Copy Markdown
Collaborator Author

@Pulkit0110 Replied to your comments, please review further. Feel free to resolve if comments sounds good.

Comment thread tests/unit/asyncio/test_async_multi_range_downloader.py
@chandra-siri chandra-siri merged commit e26888f into main Sep 29, 2025
15 checks passed
@chandra-siri chandra-siri deleted the bidi_reads_6_read_obj_stream branch September 29, 2025 07:54
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: storage Issues related to the googleapis/python-storage API. size: l Pull request size is large.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants