feat(storage): add rewrite_until_done helper as extension trait#3328
feat(storage): add rewrite_until_done helper as extension trait#3328suzmue merged 3 commits intogoogleapis:mainfrom
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3328 +/- ##
==========================================
+ Coverage 96.29% 96.32% +0.02%
==========================================
Files 113 114 +1
Lines 4510 4517 +7
==========================================
+ Hits 4343 4351 +8
+ Misses 167 166 -1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
This PR still needs tests, but I welcome thoughts on whether this is 1) the right location and 2) the right way to expose this helper. |
I really like the ergonomics of the trait extension. I think it is nicer than a fn of a builder. I am cool with it in The only thing that isn't obvious to me is whether the loop should have some sort of exhaustion policy. Like if we are making progress too slowly. I don't think we need a policy. I think even if we want such a policy, we would add it to the |
💯
We are extending builders, so
I would keep this simple. If we want a loop with exhaustion we can add another extension function (in the future) that transforms the rewrite into a stream. Then they have all the power of streams to do things like: use futures::stream::StreamExt;
let builder = client
.rewrite_object()
.set_source_bucket(format!("projects/_/buckets/{source_bucket_id}"))
.set_source_object(SOURCE_NAME)
.set_destination_bucket(format!("projects/_/buckets/{dest_bucket_id}"))
.set_destination_name(DEST_NAME);
let moved = builder
.to_stream() // A new thing
.take(100) // at most 100 attempts.
.map(|v| { /* return some error if the stream is too slow */ })
.try_fold(None, |object, response| if response.done { response.resource } else { object })A simple function like the one provided in this PR will handle 90% of the cases. We can add the streams for the next 9%, and customers can fallback to an explicit loop for the remaining 1%. |
Created
Tests are added! |
dbolduc
left a comment
There was a problem hiding this comment.
I disagree with the return type, but other than that, I just have nits
Several examples for object rewrite operations required a manual loop to poll the operation until it was complete. This boilerplate is cumbersome for users and prone to error.
This change introduces a
RewriteObjectExtextension trait, which providesrewrite_until_done()helper method on theRewriteObjectbuilder. This function encapsulates the polling logic, automatically handling the rewrite_token and looping until the operation is finished.This extension trait lives in the new
builder_extmodule. This is modeled after themodel_extmodule which contains other convenience functions and types that are built on top of generated types.For #3204