Skip to content

Cloud Storage v2.48.1: Storage.moveBlob throws exception #2925

@wsjlamoureaux

Description

@wsjlamoureaux

Description

The new MoveBlob operation fails for non-hierarchical-namespace buckets. We found that the operation fails on buckets with a flat namespace, but succeeds on buckets created with a hierarchical namespace.

The latest cloud storage documentation advertises an atomic move blob: https://cloud.google.com/java/docs/reference/google-cloud-storage/latest/com.google.cloud.storage.Storage#com_google_cloud_storage_Storage_moveBlob_com_google_cloud_storage_Storage_MoveBlobRequest_, but does not mention any restriction in its use.

Environment details

Java 21

Questions

  1. Is the documentation incomplete?
  2. Will the MoveBlob operation be supported for flat-namespace buckets?
  3. Have we missed something?

Code example

package com.example.service

import com.google.cloud.storage.Blob
import com.google.cloud.storage.BlobId
import com.google.cloud.storage.BlobInfo
import com.google.cloud.storage.Storage
import com.google.cloud.storage.StorageException
import com.google.cloud.storage.StorageOptions
import spock.lang.IgnoreIf
import spock.lang.Specification

import java.nio.charset.StandardCharsets

@IgnoreIf(
    value = {!System.getenv('GOOGLE_CLOUD_PROJECT')},
    reason = 'GOOGLE_CLOUD_PROJECT environment variable not set'
)
class MoveBlobTest extends Specification {
    private static final String FLAT_BUCKET_NAME = 'move_blob_test_flat'
    private static final String HIERARCHICAL_BUCKET_NAME = 'move_blob_test_hierarchical'

    // Relying on the GOOGLE_CLOUD_PROJECT environment variable for the project-id value
    private final Storage storage = StorageOptions.newBuilder().build().service

    def "Should fail to atomically move a blob in a flat-namespace bucket"() {
        given: "A source blob"
        Blob sourceBlob = storage.create(
            BlobInfo.newBuilder(FLAT_BUCKET_NAME, 'source-file.txt')
                .setContentType('text/plain')
                .build(),
            'file contents'.getBytes(StandardCharsets.UTF_8)
        )
        and: "A destination blob id"
        BlobId targetBlobId = BlobId.of(FLAT_BUCKET_NAME, 'target-file.txt')

        when: "A move is requested from source to target"
        Blob targetBlob = storage.moveBlob(
            Storage.MoveBlobRequest.newBuilder()
                .setSource(sourceBlob.blobId)
                .setTarget(targetBlobId)
                .build()
        )

        then: "The move operation fails with a StorageException"
        thrown(StorageException)

        and: "The source blob still exists"
        sourceBlob.exists()

        and: "The target blob does not exist"
        !targetBlob?.exists()
    }

    def "Should atomically move a blob in a hierarchical-namespace bucket"() {
        given: "A source blob"
        Blob sourceBlob = storage.create(
            BlobInfo.newBuilder(HIERARCHICAL_BUCKET_NAME, 'source-file.txt')
                .setContentType('text/plain')
                .build(),
            'file contents'.getBytes(StandardCharsets.UTF_8)
        )
        and: "A destination blob id"
        BlobId targetBlobId = BlobId.of(HIERARCHICAL_BUCKET_NAME, 'target-file.txt')

        when: "A move is requested from source to target"
        Blob targetBlob = storage.moveBlob(
            Storage.MoveBlobRequest.newBuilder()
                .setSource(sourceBlob.blobId)
                .setTarget(targetBlobId)
                .build()
        )

        then: "The move operation succeeds and returns a Blob"
        targetBlob

        and: "The source blob no longer exists"
        !sourceBlob.exists()
    }
}

Stack trace

com.google.cloud.storage.StorageException: The bucket does not support hierarchical namespace.
	at com.google.cloud.storage.StorageException.translate(StorageException.java:209)
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:347)
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.moveObject(HttpStorageRpc.java:1195)
	at com.google.cloud.storage.StorageImpl.lambda$moveBlob$73(StorageImpl.java:1711)
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:102)
	at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
	at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
	at com.google.cloud.storage.Retrying.run(Retrying.java:65)
	at com.google.cloud.storage.StorageImpl.run(StorageImpl.java:1611)
	at com.google.cloud.storage.StorageImpl.moveBlob(StorageImpl.java:1708)
	at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
	at com.example.service.MoveBlobTest.$spock_feature_0_0(MoveBlobTest.groovy:38)
        ... more
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 409 Conflict
POST https://storage.googleapis.com/storage/v1/b/move_blob_test_flat/o/source-file.txt/moveTo/o/target-file.txt
{
  "code" : 409,
  "errors" : [ {
    "domain" : "global",
    "message" : "The bucket does not support hierarchical namespace.",
    "reason" : "conflict"
  } ],
  "message" : "The bucket does not support hierarchical namespace."
}
	at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$3.interceptResponse(AbstractGoogleClientRequest.java:479)
	at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:565)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:506)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:616)
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.moveObject(HttpStorageRpc.java:1193)
	... 102 more

External references such as API reference guides

Metadata

Metadata

Assignees

Labels

api: storageIssues related to the googleapis/java-storage API.type: docsImprovement to the documentation for an API.type: questionRequest for information or clarification. Not an issue.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions