Skip to content

Commit b225fc4

Browse files
huangjiahuaYuan325
huangjiahua
and
Yuan325
authored
feat(source/cloud-storage): add write/copy/move/delete object tools (#3139)
## Description Adds four new Cloud Storage object mutation tools: - `cloud-storage-write-object` - write text content directly to a GCS object - `cloud-storage-copy-object` - copy an object within or across buckets - `cloud-storage-move-object` - atomic rename within a bucket via the native move API - `cloud-storage-delete-object` - delete a single object Coverage: - Unit tests for each new tool, including YAML parsing and Invoke validation. - Integration test config wiring for the new Cloud Storage tools. - Source and integration docs added for all four new tools. ## PR Checklist - [x] Make sure to open an issue as a bug/issue before writing your code! - [x] Ensure the tests and linter pass - [x] Code coverage does not decrease (if any source code was changed) - [x] Appropriate docs were updated (if necessary) - [x] Make sure to add `!` if this involves a breaking change ## Issue Reference Fixes # 🦕 Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
1 parent 0ee259d commit b225fc4

16 files changed

Lines changed: 2167 additions & 2 deletions

File tree

‎cmd/internal/imports.go‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,12 +83,16 @@ import (
8383
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudsqlpg/vectorassistdefinespec"
8484
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudsqlpg/vectorassistgeneratequery"
8585
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudsqlpg/vectorassistmodifyspec"
86+
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudstorage/cloudstoragecopyobject"
87+
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudstorage/cloudstoragedeleteobject"
8688
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudstorage/cloudstoragedownloadobject"
8789
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudstorage/cloudstoragegetobjectmetadata"
8890
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudstorage/cloudstoragelistbuckets"
8991
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudstorage/cloudstoragelistobjects"
92+
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudstorage/cloudstoragemoveobject"
9093
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudstorage/cloudstoragereadobject"
9194
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudstorage/cloudstorageuploadobject"
95+
_ "github.com/googleapis/mcp-toolbox/internal/tools/cloudstorage/cloudstoragewriteobject"
9296
_ "github.com/googleapis/mcp-toolbox/internal/tools/cockroachdb/cockroachdbexecutesql"
9397
_ "github.com/googleapis/mcp-toolbox/internal/tools/cockroachdb/cockroachdblistschemas"
9498
_ "github.com/googleapis/mcp-toolbox/internal/tools/cockroachdb/cockroachdblisttables"

‎docs/en/integrations/cloud-storage/source.md‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ linkTitle: "Source"
44
type: docs
55
weight: 1
66
description: >
7-
Cloud Storage is Google Cloud's managed service for storing unstructured objects (files) in buckets. Toolbox connects at the project level, allowing tools to list buckets, list objects, read object metadata and content, and transfer objects between Cloud Storage and the server filesystem.
7+
Cloud Storage is Google Cloud's managed service for storing unstructured objects (files) in buckets. Toolbox connects at the project level, allowing tools to list buckets, list objects, read object metadata and content, mutate objects, and transfer objects between Cloud Storage and the server filesystem.
88
no_list: true
99
---
1010

@@ -47,9 +47,20 @@ identity has the appropriate role for the tools being exposed. Common roles:
4747
`cloud-storage-get-object-metadata`, `cloud-storage-read-object`, and
4848
`cloud-storage-download-object`.
4949
- `roles/storage.objectUser` — read and write access to objects, sufficient for
50-
`cloud-storage-upload-object`.
50+
`cloud-storage-upload-object`, `cloud-storage-write-object`, and
51+
`cloud-storage-copy-object`.
5152
- `roles/storage.admin` — full control, including bucket management
5253

54+
Object mutation tools require the corresponding object permissions:
55+
56+
- `cloud-storage-upload-object`, `cloud-storage-write-object`, and
57+
`cloud-storage-copy-object` require object create or update permissions on
58+
the destination object.
59+
- `cloud-storage-move-object` requires `storage.objects.move` and
60+
`storage.objects.create` in the same bucket. If the destination object
61+
already exists, `storage.objects.delete` is also required.
62+
- `cloud-storage-delete-object` requires object delete permission.
63+
5364
See [Cloud Storage IAM roles][gcs-iam] for the full list.
5465

5566
Tools that read from or write to local files operate on the filesystem of the

‎docs/en/integrations/cloud-storage/tools/cloud-storage-copy-object.md‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
---
2+
title: "cloud-storage-copy-object"
3+
type: docs
4+
weight: 8
5+
description: >
6+
A "cloud-storage-copy-object" tool copies a Cloud Storage object to another object, including across buckets.
7+
---
8+
9+
## About
10+
11+
A `cloud-storage-copy-object` tool copies an object from one Cloud Storage
12+
location to another. The source and destination bucket parameters are separate,
13+
so the destination can be in the same bucket or a different bucket.
14+
15+
Existing destination objects are replaced.
16+
17+
## Compatible Sources
18+
19+
{{< compatible-sources >}}
20+
21+
## Requirements
22+
23+
The Cloud Storage credentials must be able to read the source object and create
24+
or update the destination object.
25+
26+
## Parameters
27+
28+
| **parameter** | **type** | **required** | **description** |
29+
|--------------------|:--------:|:------------:|------------------------------------------------------------------------------------|
30+
| source_bucket | string | true | Name of the Cloud Storage bucket containing the source object. |
31+
| source_object | string | true | Full source object name (path) within the source bucket, e.g. `path/to/file.txt`. |
32+
| destination_bucket | string | true | Name of the Cloud Storage bucket to copy into. |
33+
| destination_object | string | true | Full destination object name (path) within the destination bucket. |
34+
35+
## Example
36+
37+
```yaml
38+
kind: tool
39+
name: copy_object
40+
type: cloud-storage-copy-object
41+
source: my-gcs-source
42+
description: Use this tool to copy Cloud Storage objects.
43+
```
44+
45+
## Output Format
46+
47+
The tool returns a JSON object with:
48+
49+
| **field** | **type** | **description** |
50+
|--------------------|:--------:|--------------------------------------------------|
51+
| sourceBucket | string | Source Cloud Storage bucket. |
52+
| sourceObject | string | Source Cloud Storage object name. |
53+
| destinationBucket | string | Destination Cloud Storage bucket. |
54+
| destinationObject | string | Destination Cloud Storage object name. |
55+
| bytes | integer | Size of the copied object. |
56+
| contentType | string | Content type recorded on the destination object. |
57+
58+
## Reference
59+
60+
| **field** | **type** | **required** | **description** |
61+
|-------------|:--------:|:------------:|------------------------------------------------------|
62+
| type | string | true | Must be "cloud-storage-copy-object". |
63+
| source | string | true | Name of the Cloud Storage source to copy objects in. |
64+
| description | string | true | Description of the tool that is passed to the LLM. |

‎docs/en/integrations/cloud-storage/tools/cloud-storage-delete-object.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
---
2+
title: "cloud-storage-delete-object"
3+
type: docs
4+
weight: 10
5+
description: >
6+
A "cloud-storage-delete-object" tool deletes a Cloud Storage object.
7+
---
8+
9+
## About
10+
11+
A `cloud-storage-delete-object` tool deletes a single object from a Cloud
12+
Storage bucket.
13+
14+
## Compatible Sources
15+
16+
{{< compatible-sources >}}
17+
18+
## Requirements
19+
20+
The Cloud Storage credentials must be able to delete the target object.
21+
22+
## Parameters
23+
24+
| **parameter** | **type** | **required** | **description** |
25+
|---------------|:--------:|:------------:|---------------------------------------------------------------------|
26+
| bucket | string | true | Name of the Cloud Storage bucket containing the object to delete. |
27+
| object | string | true | Full object name (path) within the bucket, e.g. `path/to/file.txt`. |
28+
29+
## Example
30+
31+
```yaml
32+
kind: tool
33+
name: delete_object
34+
type: cloud-storage-delete-object
35+
source: my-gcs-source
36+
description: Use this tool to delete Cloud Storage objects.
37+
```
38+
39+
## Output Format
40+
41+
The tool returns a JSON object with:
42+
43+
| **field** | **type** | **description** |
44+
|-----------|:--------:|---------------------------------------------|
45+
| bucket | string | Cloud Storage bucket containing the object. |
46+
| object | string | Cloud Storage object name that was deleted. |
47+
| deleted | boolean | Whether the delete request completed. |
48+
49+
## Reference
50+
51+
| **field** | **type** | **required** | **description** |
52+
|-------------|:--------:|:------------:|--------------------------------------------------------|
53+
| type | string | true | Must be "cloud-storage-delete-object". |
54+
| source | string | true | Name of the Cloud Storage source to delete objects in. |
55+
| description | string | true | Description of the tool that is passed to the LLM. |

‎docs/en/integrations/cloud-storage/tools/cloud-storage-move-object.md‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
---
2+
title: "cloud-storage-move-object"
3+
type: docs
4+
weight: 9
5+
description: >
6+
A "cloud-storage-move-object" tool atomically moves or renames a Cloud Storage object within the same bucket.
7+
---
8+
9+
## About
10+
11+
A `cloud-storage-move-object` tool atomically moves or renames an object within
12+
the same Cloud Storage bucket by using Cloud Storage's native move API.
13+
14+
This tool does not perform cross-bucket moves. For a cross-bucket move, call
15+
`cloud-storage-copy-object` first, verify the destination, and then call
16+
`cloud-storage-delete-object` on the source.
17+
18+
## Compatible Sources
19+
20+
{{< compatible-sources >}}
21+
22+
## Requirements
23+
24+
The Cloud Storage credentials must have `storage.objects.move` and
25+
`storage.objects.create` permissions in the bucket. If the destination object
26+
already exists, `storage.objects.delete` is also required.
27+
28+
## Parameters
29+
30+
| **parameter** | **type** | **required** | **description** |
31+
|--------------------|:--------:|:------------:|---------------------------------------------------------------------------------|
32+
| bucket | string | true | Name of the Cloud Storage bucket containing the object to move. |
33+
| source_object | string | true | Full source object name (path) within the bucket, e.g. `path/to/file.txt`. |
34+
| destination_object | string | true | Full destination object name (path) within the same bucket. |
35+
36+
## Example
37+
38+
```yaml
39+
kind: tool
40+
name: move_object
41+
type: cloud-storage-move-object
42+
source: my-gcs-source
43+
description: Use this tool to move or rename an object within a Cloud Storage bucket.
44+
```
45+
46+
## Output Format
47+
48+
The tool returns a JSON object with:
49+
50+
| **field** | **type** | **description** |
51+
|--------------------|:--------:|------------------------------------------------|
52+
| bucket | string | Cloud Storage bucket containing the object. |
53+
| sourceObject | string | Original object name. |
54+
| destinationObject | string | Destination object name. |
55+
| bytes | integer | Size of the moved object. |
56+
| contentType | string | Content type recorded on the destination object. |
57+
58+
## Reference
59+
60+
| **field** | **type** | **required** | **description** |
61+
|-------------|:--------:|:------------:|------------------------------------------------------|
62+
| type | string | true | Must be "cloud-storage-move-object". |
63+
| source | string | true | Name of the Cloud Storage source to move objects in. |
64+
| description | string | true | Description of the tool that is passed to the LLM. |

‎docs/en/integrations/cloud-storage/tools/cloud-storage-write-object.md‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
---
2+
title: "cloud-storage-write-object"
3+
type: docs
4+
weight: 7
5+
description: >
6+
A "cloud-storage-write-object" tool writes text content directly to a Cloud Storage object.
7+
---
8+
9+
## About
10+
11+
A `cloud-storage-write-object` tool writes text content from the tool request
12+
directly into a Cloud Storage object. It is useful for creating or replacing
13+
small text objects without first writing a local file on the Toolbox server.
14+
15+
When `content_type` is empty, Cloud Storage detects the content type from the
16+
written bytes.
17+
18+
## Compatible Sources
19+
20+
{{< compatible-sources >}}
21+
22+
## Requirements
23+
24+
The Cloud Storage credentials must be able to create or update the target
25+
object.
26+
27+
## Parameters
28+
29+
| **parameter** | **type** | **required** | **description** |
30+
|---------------|:--------:|:------------:|-------------------------------------------------------------------------------------------------------|
31+
| bucket | string | true | Name of the Cloud Storage bucket to write into. |
32+
| object | string | true | Full object name (path) within the bucket, e.g. `path/to/file.txt`. |
33+
| content | string | true | Text content to write to the Cloud Storage object. |
34+
| content_type | string | false | MIME type to record on the written object. When empty, Cloud Storage auto-detects from the content. |
35+
36+
## Example
37+
38+
```yaml
39+
kind: tool
40+
name: write_object
41+
type: cloud-storage-write-object
42+
source: my-gcs-source
43+
description: Use this tool to write text content to Cloud Storage.
44+
```
45+
46+
## Output Format
47+
48+
The tool returns a JSON object with:
49+
50+
| **field** | **type** | **description** |
51+
|-------------|:--------:|----------------------------------------------|
52+
| bucket | string | Cloud Storage bucket that received content. |
53+
| object | string | Cloud Storage object name that was written. |
54+
| bytes | integer | Number of bytes written. |
55+
| contentType | string | Content type recorded on the written object. |
56+
57+
## Reference
58+
59+
| **field** | **type** | **required** | **description** |
60+
|-------------|:--------:|:------------:|------------------------------------------------------|
61+
| type | string | true | Must be "cloud-storage-write-object". |
62+
| source | string | true | Name of the Cloud Storage source to write objects to. |
63+
| description | string | true | Description of the tool that is passed to the LLM. |

‎internal/sources/cloudstorage/cloudstorage.go‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -315,6 +315,91 @@ func (s *Source) UploadObject(ctx context.Context, bucket, object, source, conte
315315
}, nil
316316
}
317317

318+
// WriteObject writes text content directly into a GCS object. When contentType
319+
// is empty, the writer's ContentType is left unset so Cloud Storage detects it
320+
// from the first 512 bytes. The returned contentType is the post-Close value
321+
// from w.Attrs(), i.e. what GCS actually recorded.
322+
func (s *Source) WriteObject(ctx context.Context, bucket, object, content, contentType string) (map[string]any, error) {
323+
w := s.client.Bucket(bucket).Object(object).NewWriter(ctx)
324+
if contentType != "" {
325+
w.ContentType = contentType
326+
}
327+
328+
n, err := io.WriteString(w, content)
329+
if err != nil {
330+
_ = w.Close()
331+
return nil, fmt.Errorf("failed to write content to object %q in bucket %q: %w", object, bucket, err)
332+
}
333+
if err := w.Close(); err != nil {
334+
return nil, fmt.Errorf("failed to finalize write to %q/%q: %w", bucket, object, err)
335+
}
336+
337+
attrs := w.Attrs()
338+
finalContentType := ""
339+
if attrs != nil {
340+
finalContentType = attrs.ContentType
341+
}
342+
return map[string]any{
343+
"bucket": bucket,
344+
"object": object,
345+
"bytes": n,
346+
"contentType": finalContentType,
347+
}, nil
348+
}
349+
350+
// CopyObject copies an object to a destination object. The destination may be
351+
// in the same bucket or a different bucket. Existing destination objects are
352+
// replaced, matching Cloud Storage's copy semantics without preconditions.
353+
func (s *Source) CopyObject(ctx context.Context, sourceBucket, sourceObject, destinationBucket, destinationObject string) (map[string]any, error) {
354+
src := s.client.Bucket(sourceBucket).Object(sourceObject)
355+
dst := s.client.Bucket(destinationBucket).Object(destinationObject)
356+
357+
attrs, err := dst.CopierFrom(src).Run(ctx)
358+
if err != nil {
359+
return nil, fmt.Errorf("failed to copy %q/%q to %q/%q: %w", sourceBucket, sourceObject, destinationBucket, destinationObject, err)
360+
}
361+
362+
return map[string]any{
363+
"sourceBucket": sourceBucket,
364+
"sourceObject": sourceObject,
365+
"destinationBucket": destinationBucket,
366+
"destinationObject": destinationObject,
367+
"bytes": attrs.Size,
368+
"contentType": attrs.ContentType,
369+
}, nil
370+
}
371+
372+
// MoveObject atomically renames or moves an object within the same bucket using
373+
// Cloud Storage's native move API. Cross-bucket moves should be modeled as
374+
// CopyObject followed by DeleteObject.
375+
func (s *Source) MoveObject(ctx context.Context, bucket, sourceObject, destinationObject string) (map[string]any, error) {
376+
attrs, err := s.client.Bucket(bucket).Object(sourceObject).Move(ctx, storage.MoveObjectDestination{Object: destinationObject})
377+
if err != nil {
378+
return nil, fmt.Errorf("failed to move %q to %q in bucket %q: %w", sourceObject, destinationObject, bucket, err)
379+
}
380+
381+
return map[string]any{
382+
"bucket": bucket,
383+
"sourceObject": sourceObject,
384+
"destinationObject": destinationObject,
385+
"bytes": attrs.Size,
386+
"contentType": attrs.ContentType,
387+
}, nil
388+
}
389+
390+
// DeleteObject deletes a GCS object.
391+
func (s *Source) DeleteObject(ctx context.Context, bucket, object string) (map[string]any, error) {
392+
if err := s.client.Bucket(bucket).Object(object).Delete(ctx); err != nil {
393+
return nil, fmt.Errorf("failed to delete object %q in bucket %q: %w", object, bucket, err)
394+
}
395+
396+
return map[string]any{
397+
"bucket": bucket,
398+
"object": object,
399+
"deleted": true,
400+
}, nil
401+
}
402+
318403
func initGCSClient(ctx context.Context, tracer trace.Tracer, name, project string) (*storage.Client, error) {
319404
//nolint:all // Reassigned ctx
320405
ctx, span := sources.InitConnectionSpan(ctx, tracer, SourceType, name)

0 commit comments

Comments
 (0)