Skip to content

feat(gateway): add artifact RPCs#74769

Closed
volcano303 wants to merge 1 commit intoopenclaw:mainfrom
volcano303:fix/74706-artifacts-rpc
Closed

feat(gateway): add artifact RPCs#74769
volcano303 wants to merge 1 commit intoopenclaw:mainfrom
volcano303:fix/74706-artifacts-rpc

Conversation

@volcano303
Copy link
Copy Markdown
Contributor

Summary

Describe the problem and fix in 2–5 bullets:

  • Problem: Gateway clients have no supported artifact RPC surface, so SDK/app clients must scrape chat history and infer downloadable media themselves.
  • Why it matters: External app clients need stable artifact discovery and download semantics for generated or attached media without depending on transcript internals.
  • What changed: Added artifacts.list, artifacts.get, and artifacts.download gateway RPCs over session transcript artifact blocks, with protocol schemas, method discovery, read-scope authorization, and typed unsupported/not-found errors.
  • What did NOT change (scope boundary): This does not add a new artifact store or expose arbitrary local file paths; unsupported artifact sources return explicit unsupported errors.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Root Cause (if applicable)

  • Root cause: Artifacts existed only as transcript content blocks, with no supported gateway RPC projection for SDK/app clients.
  • Missing detection / guardrail: There was no focused gateway test asserting artifact list/get/download behavior or typed missing/unsupported errors.
  • Contributing context (if known): The SDK happy path needs artifact access without requiring clients to parse chat history payload details.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: src/gateway/server-methods/artifacts.test.ts
  • Scenario the test should lock in: Session transcript media blocks are listed as stable artifact summaries, can be fetched by ID, can be downloaded as base64 bytes, and missing/unsupported query paths return typed errors.
  • Why this is the smallest reliable guardrail: The new behavior is a gateway RPC projection over existing transcript state, so handler-level tests cover the API contract directly.
  • Existing test that already covers this (if any): N/A
  • If no new test is added, why not: N/A

User-visible / Behavior Changes

SDK and app clients can call artifacts.list, artifacts.get, and artifacts.download with sessionKey, runId, or taskId query context. Inline transcript artifacts download as base64 bytes; URL-backed artifacts return a URL; unsupported sources return a typed error.

Diagram (if applicable)

Before:
SDK client -> chat.history -> parse transcript media internals

After:
SDK client -> artifacts.list/get/download -> normalized artifact contract

Security Impact (required)

  • New permissions/capabilities? (No)
  • Secrets/tokens handling changed? (No)
  • New/changed network calls? (No)
  • Command/tool execution surface changed? (No)
  • Data access scope changed? (Yes)
  • If any Yes, explain risk + mitigation: Artifact RPCs expose only artifacts already present in authorized chat/session transcript data and are gated by existing operator.read scope. They do not read arbitrary local file paths.

Repro + Verification

Environment

  • OS: Linux
  • Runtime/container: Node 22.14.0, pnpm
  • Model/provider: N/A
  • Integration/channel (if any): Gateway RPC
  • Relevant config (redacted): Default test config

Steps

  1. Seed a session transcript message with an image/file content block.
  2. Call artifacts.list with the session key.
  3. Call artifacts.get and artifacts.download with the returned artifact ID.
  4. Call with missing/unknown artifact context.

Expected

  • Artifact summaries are stable and omit raw bytes.
  • artifacts.download returns base64 bytes or URL depending on source.
  • Missing/unsupported cases return typed errors.

Actual

  • Matches expected after this PR.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios: artifacts.list, artifacts.get, artifacts.download, runId-to-session lookup, missing query scope, missing artifact ID, protocol schema compilation, method-scope discovery.
  • Edge cases checked: Inline base64 artifact bytes are omitted from summaries and returned only from download.
  • What you did not verify: A live SDK/OpenMeow client against a running gateway.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

If a bot review conversation is addressed by this PR, resolve that conversation yourself. Do not leave bot review conversation cleanup for maintainers.

Compatibility / Migration

  • Backward compatible? (Yes)
  • Config/env changes? (No)
  • Migration needed? (No)
  • If yes, exact upgrade steps: N/A

Risks and Mitigations

  • Risk: The first artifact source is intentionally conservative and may return artifact_download_unsupported for transcript blocks that reference local-only paths.
    • Mitigation: This avoids exposing arbitrary local filesystem paths while establishing the stable RPC contract for inline and URL-backed artifacts.

@openclaw-barnacle openclaw-barnacle Bot added app: web-ui App: web-ui gateway Gateway runtime size: L labels Apr 30, 2026
@clawsweeper
Copy link
Copy Markdown
Contributor

clawsweeper Bot commented Apr 30, 2026

Thanks for the context here. I swept through the related work, and this is now duplicate or superseded.

Close as superseded. The artifact RPC feature is still valid and not implemented on current main, but this draft branch has blocking protocol, coverage, generated-client, test-shape, and download-source issues, and the same remaining work is now tracked by the newer open replacement PR #74898.

So I’m closing this here and keeping the remaining discussion on the canonical linked item.

Review details

Best possible solution:

Continue the artifact RPC work on the canonical replacement branch or another maintainer-selected branch, fold in the remaining protocol/security fixes, and land it with Gateway, SDK, generated-client, and docs coverage aligned.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main reproduces the missing feature by checking that the SDK artifact helpers throw unsupported errors and Gateway method discovery lacks artifacts.* methods; the PR-specific failures are source-verifiable through the TypeBox schema composition and existing input_image/image_url transcript blocks.

Is this the best way to solve the issue?

No. The direction is right, but this exact draft is not the best merge vehicle because it is superseded by #74898 and still has blocking validator, media-shape, generated-client, test type, and URL-source issues.

Security review:

Security review needs attention: The PR adds a read-scoped artifact download surface and currently treats any non-data URL-like transcript value as downloadable.

  • [medium] Validate artifact URL download sources — src/gateway/server-methods/artifacts.ts:129
    resolveBlockDownload classifies any non-data url, openUrl, or source.url as a URL download. Since this reads raw transcript blocks, unsupported schemes or local-path-like values could be surfaced to read-scoped clients despite the stated boundary that unsupported local-only sources return typed unsupported errors.
    Confidence: 0.82

What I checked:

  • Current main lacks artifact Gateway methods: On current main, method discovery and read scopes include agents.files entries but no artifacts.list, artifacts.get, or artifacts.download; the SDK artifact namespace still throws explicit unsupported errors. (src/gateway/server-methods-list.ts:71, c4a4c189f11e)
  • Superseding PR exists: The provided timeline shows feat(gateway): add artifact RPCs #74898 cross-referenced this PR at 2026-04-30T06:16:01Z, then this PR was converted to draft at 2026-04-30T06:17:36Z. The feat(gateway): add artifact RPCs #74898 patch starts with the same 684-line artifact RPC commit and adds a second protocol-alignment commit with generated Swift models and the flat get/download schema. (93ad914d6da3)
  • TypeBox intersect behavior breaks get/download validation: The PR uses Type.Intersect for locator+artifactId params, while typebox 1.1.34 emits allOf for Intersect. With additionalProperties:false on both operands, Ajv rejects real params containing both artifactId and sessionKey/runId/taskId. (src/gateway/protocol/schema/artifacts.ts:13, 385c5c5af66f)
  • Existing transcript media shape is missed: Current main writes transcript display blocks as { type: "input_image", image_url }, but the PR artifact detector only accepts image/audio/file types or url/openUrl/data/source fields. (src/gateway/server-methods/chat-webchat-media.ts:235, c4a4c189f11e)
  • Protocol generated outputs are required: The protocol check regenerates and diffs Swift GatewayModels outputs when ProtocolSchemas changes; this PR adds protocol schemas but its changed-file list does not include the generated Swift files. (package.json:1418, c4a4c189f11e)
  • Review context already identified blockers: The existing ClawSweeper review comment on this PR called out the closed schema issue, missing image_url handling, stale generated protocol models, and URL-source filtering as pre-merge blockers. (385c5c5af66f)

Likely related people:

  • @steipete: Local blame/log history for the central Gateway protocol, method list, SDK artifact namespace, and WebChat media files points to Peter Steinberger’s recent main-branch commits in this shallow checkout. (role: recent maintainer; confidence: medium; commits: c976cf6ebd8d, cbc2ba093146; files: src/gateway/protocol/schema/protocol-schemas.ts, src/gateway/server-methods-list.ts, src/gateway/method-scopes.ts)
  • @BunsDev: The linked maintainer issue Gateway RPC: add SDK-facing artifacts APIs #74706 defines the SDK-facing artifacts.list/get/download acceptance criteria this PR attempts to implement, and SDK: stabilize app-client happy path for agents, sessions, runs #74704 frames the broader SDK app-client happy path. (role: linked API requester; confidence: medium; files: docs/reference/openclaw-sdk-api-design.md, docs/concepts/openclaw-sdk.md)
  • @tmimmanuel: Authored the newer open feat(gateway): add artifact RPCs #74898 replacement branch that starts from this PR’s artifact RPC diff and adds protocol-alignment fixes, making it the likely current implementation thread. (role: adjacent follow-up owner; confidence: medium; commits: 2a510667420c, 93ad914d6da3; files: src/gateway/protocol/schema/artifacts.ts, src/gateway/server-methods/artifacts.ts, apps/macos/Sources/OpenClawProtocol/GatewayModels.swift)

Codex review notes: model gpt-5.5, reasoning high; reviewed against c4a4c189f11e.

@clawsweeper clawsweeper Bot mentioned this pull request Apr 30, 2026
25 tasks
@volcano303 volcano303 marked this pull request as draft April 30, 2026 06:17
@clawsweeper clawsweeper Bot closed this Apr 30, 2026
BunsDev added a commit that referenced this pull request May 1, 2026
Add Gateway artifact RPCs and SDK helpers for list/get/download, with transcript provenance checks, safer download source handling, task/run/session coverage, generated protocol models, docs, and the refreshed generated config schema baseline.

Closes #74706.
Refs #74898, #74769, #74804, #74786.
lxe pushed a commit to lxe/openclaw that referenced this pull request May 6, 2026
Add Gateway artifact RPCs and SDK helpers for list/get/download, with transcript provenance checks, safer download source handling, task/run/session coverage, generated protocol models, docs, and the refreshed generated config schema baseline.

Closes openclaw#74706.
Refs openclaw#74898, openclaw#74769, openclaw#74804, openclaw#74786.
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 9, 2026
Add Gateway artifact RPCs and SDK helpers for list/get/download, with transcript provenance checks, safer download source handling, task/run/session coverage, generated protocol models, docs, and the refreshed generated config schema baseline.

Closes openclaw#74706.
Refs openclaw#74898, openclaw#74769, openclaw#74804, openclaw#74786.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

app: web-ui App: web-ui gateway Gateway runtime size: L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gateway RPC: add SDK-facing artifacts APIs

1 participant