feat(firestore): adds Bulkwriter support to Firestore client#5946
Merged
feat(firestore): adds Bulkwriter support to Firestore client#5946
Conversation
enocom
reviewed
Jul 18, 2022
Contributor
enocom
left a comment
There was a problem hiding this comment.
One last thing about closing the bulk writer.
enocom
approved these changes
Jul 19, 2022
Contributor
|
Merge-on-green attempted to merge your PR for 6 hours, but it was not mergeable because either one of your required status checks failed, one of your required reviews was not approved, or there is a do not merge label. Learn more about your required status checks here: https://help.github.com/en/github/administering-a-repository/enabling-required-status-checks. You can remove and reapply the label to re-run the bot. |
enocom
reviewed
Jul 20, 2022
| // After calling End(), calling any additional method automatically returns | ||
| // with an error. This method completes when there are no more pending writes | ||
| // in the queue. | ||
| func (bw *BulkWriter) End() { |
Contributor
There was a problem hiding this comment.
I think there might be a minor race condition here. Here's the idea:
- Goroutine 1 in
checkWriteConditionacquires the lock, sees thatisOpenis true, and then releases the lock - Goroutine 2 in
Endacquires the lock, setsisOpento false, releases the lock, and then flushes - Goroutine 1 meanwhile starts a write
The question is: does the write succeed or not in this case?
One possible option would be to use a read write mutex, where the read mutex wraps the entire write attempt and the write mutex wraps isOpen here in End.
Contributor
Author
There was a problem hiding this comment.
Very good point.
Here's what I've done to address this:
- Changed
BulkWriter.isOpenLockto aRWMutex. - Added a
Lock()and deferredUnlock()at the top of each DB mutator/write function (Create(),Delete(), etc).
Each mutator will hold the lock while queueing up the write for the Bundler. If End() is called during this time, it will will block while the mutator holds the lock.
enocom
approved these changes
Jul 21, 2022
enocom
approved these changes
Jul 21, 2022
noahdietz
added a commit
that referenced
this pull request
Jul 21, 2022
* chore: release main (#6351) * test(profiler): compile busybench before running backoff test (#6375) * chore(bigquery/storage/managedwriter): augment test logging (#6373) * chore(storage): RewriteObject implementation (#6313) * chore(storage): RewriteObject implementation * address feedback * refactor source/destination object types * address feedback * address feedback * fix test * chore(main): release storage 1.24.0 (#6336) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Chris Cotter <cjcotter@google.com> * chore(internal/gapicgen): update microgen v0.31.2 (#6383) Only includes fixes to regapic generation. * test(bigquery/storage/managedwriter): relax error checking (#6385) When a user issues a large request, the response from the backend is a bare "InvalidArgument". This PR removes additional validation on information that is only attached when interrogating the backend from a known client; it's stripped in the normal case. Internal issue 239740070 was created to address the unactionable nature of the response. Fixes: #6361 * feat(firestore): adds Bulkwriter support to Firestore client (#5946) * feat: adds Bulkwriter support to Firestore client * test(storage): unflake TestIntegration_ACL (#6392) Few minor changes to make sure potentially flaky and/or eventually consistent operations for ACLs are retried appropriately. Fixes #6379 Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Amarin (Um) Phaosawasdi <amchiclet@users.noreply.github.com> Co-authored-by: shollyman <shollyman@google.com> Co-authored-by: Noah Dietz <noahdietz@users.noreply.github.com> Co-authored-by: Chris Cotter <cjcotter@google.com> Co-authored-by: Eric Schmidt <erschmid@google.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds support for BulkWriter to the Golang client.