-
Notifications
You must be signed in to change notification settings - Fork 89
Closed
Labels
api: storageIssues related to the googleapis/java-storage API.Issues related to the googleapis/java-storage API.type: docsImprovement to the documentation for an API.Improvement to the documentation for an API.type: questionRequest for information or clarification. Not an issue.Request for information or clarification. Not an issue.
Description
Description
The new MoveBlob operation fails for non-hierarchical-namespace buckets. We found that the operation fails on buckets with a flat namespace, but succeeds on buckets created with a hierarchical namespace.
The latest cloud storage documentation advertises an atomic move blob: https://cloud.google.com/java/docs/reference/google-cloud-storage/latest/com.google.cloud.storage.Storage#com_google_cloud_storage_Storage_moveBlob_com_google_cloud_storage_Storage_MoveBlobRequest_, but does not mention any restriction in its use.
Environment details
Java 21
Questions
- Is the documentation incomplete?
- Will the MoveBlob operation be supported for flat-namespace buckets?
- Have we missed something?
Code example
package com.example.service
import com.google.cloud.storage.Blob
import com.google.cloud.storage.BlobId
import com.google.cloud.storage.BlobInfo
import com.google.cloud.storage.Storage
import com.google.cloud.storage.StorageException
import com.google.cloud.storage.StorageOptions
import spock.lang.IgnoreIf
import spock.lang.Specification
import java.nio.charset.StandardCharsets
@IgnoreIf(
value = {!System.getenv('GOOGLE_CLOUD_PROJECT')},
reason = 'GOOGLE_CLOUD_PROJECT environment variable not set'
)
class MoveBlobTest extends Specification {
private static final String FLAT_BUCKET_NAME = 'move_blob_test_flat'
private static final String HIERARCHICAL_BUCKET_NAME = 'move_blob_test_hierarchical'
// Relying on the GOOGLE_CLOUD_PROJECT environment variable for the project-id value
private final Storage storage = StorageOptions.newBuilder().build().service
def "Should fail to atomically move a blob in a flat-namespace bucket"() {
given: "A source blob"
Blob sourceBlob = storage.create(
BlobInfo.newBuilder(FLAT_BUCKET_NAME, 'source-file.txt')
.setContentType('text/plain')
.build(),
'file contents'.getBytes(StandardCharsets.UTF_8)
)
and: "A destination blob id"
BlobId targetBlobId = BlobId.of(FLAT_BUCKET_NAME, 'target-file.txt')
when: "A move is requested from source to target"
Blob targetBlob = storage.moveBlob(
Storage.MoveBlobRequest.newBuilder()
.setSource(sourceBlob.blobId)
.setTarget(targetBlobId)
.build()
)
then: "The move operation fails with a StorageException"
thrown(StorageException)
and: "The source blob still exists"
sourceBlob.exists()
and: "The target blob does not exist"
!targetBlob?.exists()
}
def "Should atomically move a blob in a hierarchical-namespace bucket"() {
given: "A source blob"
Blob sourceBlob = storage.create(
BlobInfo.newBuilder(HIERARCHICAL_BUCKET_NAME, 'source-file.txt')
.setContentType('text/plain')
.build(),
'file contents'.getBytes(StandardCharsets.UTF_8)
)
and: "A destination blob id"
BlobId targetBlobId = BlobId.of(HIERARCHICAL_BUCKET_NAME, 'target-file.txt')
when: "A move is requested from source to target"
Blob targetBlob = storage.moveBlob(
Storage.MoveBlobRequest.newBuilder()
.setSource(sourceBlob.blobId)
.setTarget(targetBlobId)
.build()
)
then: "The move operation succeeds and returns a Blob"
targetBlob
and: "The source blob no longer exists"
!sourceBlob.exists()
}
}Stack trace
com.google.cloud.storage.StorageException: The bucket does not support hierarchical namespace.
at com.google.cloud.storage.StorageException.translate(StorageException.java:209)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:347)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.moveObject(HttpStorageRpc.java:1195)
at com.google.cloud.storage.StorageImpl.lambda$moveBlob$73(StorageImpl.java:1711)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:102)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
at com.google.cloud.storage.Retrying.run(Retrying.java:65)
at com.google.cloud.storage.StorageImpl.run(StorageImpl.java:1611)
at com.google.cloud.storage.StorageImpl.moveBlob(StorageImpl.java:1708)
at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at com.example.service.MoveBlobTest.$spock_feature_0_0(MoveBlobTest.groovy:38)
... more
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 409 Conflict
POST https://storage.googleapis.com/storage/v1/b/move_blob_test_flat/o/source-file.txt/moveTo/o/target-file.txt
{
"code" : 409,
"errors" : [ {
"domain" : "global",
"message" : "The bucket does not support hierarchical namespace.",
"reason" : "conflict"
} ],
"message" : "The bucket does not support hierarchical namespace."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$3.interceptResponse(AbstractGoogleClientRequest.java:479)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:565)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:506)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:616)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.moveObject(HttpStorageRpc.java:1193)
... 102 more
External references such as API reference guides
Metadata
Metadata
Assignees
Labels
api: storageIssues related to the googleapis/java-storage API.Issues related to the googleapis/java-storage API.type: docsImprovement to the documentation for an API.Improvement to the documentation for an API.type: questionRequest for information or clarification. Not an issue.Request for information or clarification. Not an issue.