Skip to content

feature: utilize new private interface to pass 'command' through transfer manager#1096

Merged
andrewsg merged 27 commits intogoogleapis:tm-metricsfrom
MiaCY:filename_handling
Sep 15, 2023
Merged

feature: utilize new private interface to pass 'command' through transfer manager#1096
andrewsg merged 27 commits intogoogleapis:tm-metricsfrom
MiaCY:filename_handling

Conversation

@MiaCY
Copy link
Contributor

@MiaCY MiaCY commented Jul 26, 2023 •
edited
Loading

Download:

  • Create new method blob._handle_filename_and_download()
  • Refactor business logic of blob.download_to_filename() into blob._handle_filename_and_download()
  • Refactor blob.download_to_filename() to call blob._handle_filename_and_download()
  • Refactor transfer_manager.download_many() to call blob._handle_filename_and_download() or blob._prep_and_do_download()
  • Add 'command' to download_kwargs of transfer_manager.download_many() and transfer_manager.download_chunks_concurrently()

Upload:

  • Create new method blob._handle_filename_and_upload()
  • Refactor business logic of blob.upload_from_filename() into blob._handle_filename_and_upload()
  • Refactor blob.upload_from_filename() to call blob._handle_filename_and_upload()
  • Refactor transfer_manager.upload_many() to call blob._handle_filename_and_upload() or blob._prep_and _do_upload()
  • Addd 'command' to upload_kwargs of transfer_manager.upload_many()

Other:

  • Refactor transfer manager unit tests accordingly
  • Edit docstrings of related methods

MiaCY and others added 23 commits May 25, 2023 10:03
@MiaCY
Refactor client.download_blob_to_file
c528317
@MiaCY
Merge branch 'tm-metrics' of https://github.com/googleapis/python-sto…
e534e5c 
…rage into transfer_manager_usage_metrics
@MiaCY
Chore: clean up code
fdb60dd
@MiaCY
refactor blob and client unit tests
9c67a41
@MiaCY
Merge branch 'main' of https://github.com/googleapis/python-storage i…
03a9ef1 
…nto transfer_manager_usage_metrics
@MiaCY
lint reformat
0d2d825
@MiaCY
Rename _prep_and_do_download
2d0ba3b
@MiaCY
Refactor blob.upload_from_file
b41c38d
@MiaCY
Merge branch 'tm-metrics' of https://github.com/googleapis/python-sto…
a2abf20 
…rage into transfer_manager_usage_metrics
@MiaCY
Lint reformat
8b2897e
@MiaCY
feature: add 'command' argument to private upload/download interface
6cfc5ba
@MiaCY
Merge branch 'tm-metrics' into transfer_manager_usage_metrics
3b8503c
@MiaCY
lint reformat
a7927c5
@MiaCY
Merge branch 'transfer_manager_usage_metrics' of https://github.com/M…
fa681d8 
…iaCY/python-storage into transfer_manager_usage_metrics
@MiaCY
Merge branch 'main' of https://github.com/MiaCY/python-storage into t…
0487ef7 
…ransfer_manager_usage_metrics
@MiaCY
Merge branch 'tm-metrics' into transfer_manager_usage_metrics
8ac003a
@MiaCY
reduce duplication and edit docstring
31561b4
@MiaCY
Merge branch 'transfer_manager_usage_metrics' of https://github.com/M…
a834f01 
…iaCY/python-storage into transfer_manager_usage_metrics
@MiaCY
Merge branch 'tm-metrics' of https://github.com/googleapis/python-sto…
d57b49b 
…rage into transfer_manager_usage_metrics
@MiaCY
feature: utilize new private interface to pass 'command' through tran…
0e9ffb4 
…sfer manager
@MiaCY
Merge branch 'transfer_manager_usage_metrics' of https://github.com/M…
ded6af4 
…iaCY/python-storage into filename_handling
@MiaCY
add default value to content_type in blob.handle_filename_and_upload()
0a70e4c
@MiaCY
Merge branch 'transfer_manager_usage_metrics' of https://github.com/M…
9d33280 
…iaCY/python-storage into filename_handling
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: storage Issues related to the googleapis/python-storage API. labels Jul 26, 2023
@MiaCY MiaCY added kokoro:run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Jul 26, 2023
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 26, 2023
@MiaCY MiaCY removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Jul 26, 2023
@MiaCY MiaCY marked this pull request as ready for review July 26, 2023 23:30
@MiaCY MiaCY requested a review from a team as a code owner July 26, 2023 23:30
@MiaCY MiaCY requested a review from a team July 26, 2023 23:30
andrewsg
andrewsg requested changes Jul 26, 2023
View reviewed changes
cojenco
cojenco reviewed Jul 28, 2023
View reviewed changes
Copy link
Contributor

@cojenco cojenco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks Mia!

@andrewsg andrewsg merged commit 50b4bbb into googleapis:tm-metrics Sep 15, 2023
andrewsg pushed a commit that referenced this pull request Sep 18, 2023
@MiaCY @andrewsg
feature: utilize new private interface to pass 'command' through tran…
6b4ca0a 
…sfer manager (#1096)

* Refactor client.download_blob_to_file

* Chore: clean up code

* refactor blob and client unit tests

* lint reformat

* Rename _prep_and_do_download

* Refactor blob.upload_from_file

* Lint reformat

* feature: add 'command' argument to private upload/download interface

* lint reformat

* reduce duplication and edit docstring

* feature: utilize new private interface to pass 'command' through transfer manager

* add default value to content_type in blob.handle_filename_and_upload()

* add 'command' to downloan_chunks_concurrently

* docstring editing

* refactor transfer manager unit tests

* refactor expected download/upload kwargs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewsg andrewsg andrewsg approved these changes

+1 more reviewer

@cojenco cojenco cojenco approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

api: storage Issues related to the googleapis/python-storage API. size: m Pull request size is medium.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@MiaCY @andrewsg @cojenco