Consolidate membrane VM hot-patches onto fork-main (Veo + companions)#5
Conversation
Consolidates the still-needed local /opt/openclaw hot-patches onto fork-main (= upstream 94db48d + native audio #4) so the membrane VM can cut over to the fork and ship native audio ingestion. These three files were untouched by upstream in the 99d96c1→94db48d0 window, so they transplant verbatim: - extensions/google/video-generation-provider.ts — the openclaw#172 Vertex REST-bearer bypass (load-bearing per tulgey#194; SDK auth path is the openclaw#175 bug) + the #3 default-1080p resolution. - extensions/google/generation-provider-metadata.ts — Veo companion. - src/cli/program/message/register.send.ts — companion. Dropped: the session-lock patch (openclaw#195) — upstream made waitForSessionEventQueue a no-op by 94db48d, so it is obsolete. Deferred (fast-follow, refs tulgey#218): src/auto-reply/dispatch.ts (the ADR 0015 inbound-message-sequencing coalescing rewrite) and src/infra/dotenv.ts — both conflict structurally with fork-main and need a careful port + review. Refs imperfect-co/tulgey#218. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThis PR upgrades Google video generation models from preview to release versions, adds Vertex AI OAuth authentication with model aliasing, extends REST-based generation with configurable polling and richer request payloads (including image/video bytes), and introduces a CLI flag for voice note formatting in message sends. ChangesGoogle Video Generation Provider Enhancements
CLI Voice Note Support
🎯 3 (Moderate) | ⏱️ ~25 minutes
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Consolidates the still-needed local /opt/openclaw hot-patches onto fork-main (= upstream 94db48d + native audio #4) so the membrane VM can cut over to the fork and ship native audio ingestion. These three files were untouched by upstream in the 99d96c1→94db48d0 window, so they transplant verbatim: - extensions/google/video-generation-provider.ts — the openclaw#172 Vertex REST-bearer bypass (load-bearing per tulgey#194; SDK auth path is the openclaw#175 bug) + the #3 default-1080p resolution. - extensions/google/generation-provider-metadata.ts — Veo companion. - src/cli/program/message/register.send.ts — companion. Dropped: the session-lock patch (openclaw#195) — upstream made waitForSessionEventQueue a no-op by 94db48d, so it is obsolete. Deferred (fast-follow, refs tulgey#218): src/auto-reply/dispatch.ts (the ADR 0015 inbound-message-sequencing coalescing rewrite) and src/infra/dotenv.ts — both conflict structurally with fork-main and need a careful port + review. Refs imperfect-co/tulgey#218. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lint:extensions:bundled lints the whole extensions/google package, so these errors (introduced with the Veo REST fallback in #5, never linted since no later PR touched the package) block any PR that touches the extension. Surfaced by the native-audio-output change. - resolveVertexOAuthToken: brace the metadata-token if, type res.json() as { access_token?: string } (drops the unnecessary `as any`), and omit the unused catch binding. - brace the "Force rest fallback for Vertex" guard. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…aw#247) (#17) * feat(speech): native audio output via Vertex ADC route (tulgey openclaw#247) The Google speech provider already emits native generateContent AUDIO (gemini-3.1-flash-tts-preview, responseModalities:['AUDIO'] + speechConfig) and already transcodes to opus-in-ogg for voice-note delivery. The only gap was auth: it knew the AI-Studio key route only and threw "Google API key missing" on a keyless Vertex deployment (tulgey #10). This adds the Vertex ADC route so native output is the primary path on the deployment. - Add a Vertex ADC synthesis route (synthesizeGoogleVertexTtsPcm) that rides resolveGoogleVertexAuthorizedUserHeaders (the same ADC bearer the Google chat/Veo paths use), POSTing to aiplatform.googleapis.com/v1/projects/{P}/locations/{global}/publishers/ google/models/{model}:generateContent. Body, PCM extraction, WAV-wrap, and opus transcode are shared verbatim with the AI-Studio route. - Route selection (resolveGoogleTtsPcm): AI-Studio key route stays primary; fall to the Vertex ADC route when no key but ADC is present; throw with neither so the speech provider-order fallback (Cloud TTS -> text) trips on a detected failure, never a silent degrade (ADR 0024 clause 2). - isConfigured is now ADC-aware so the provider is selected keyless. - Extract buildGoogleSpeechGenerateContentBody (shared by both routes). - Test: Vertex generateContent URL shape (global + regional). Implements the membrane row of tulgey#247 / ADR 0024. Existing AI-Studio tests unaffected (real keys take the unchanged route). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(google): clear pre-existing oxlint errors in the Veo provider lint:extensions:bundled lints the whole extensions/google package, so these errors (introduced with the Veo REST fallback in #5, never linted since no later PR touched the package) block any PR that touches the extension. Surfaced by the native-audio-output change. - resolveVertexOAuthToken: brace the metadata-token if, type res.json() as { access_token?: string } (drops the unnecessary `as any`), and omit the unused catch binding. - brace the "Force rest fallback for Vertex" guard. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(speech): route Vertex TTS through guarded postJsonRequest The new Vertex ADC route used a raw fetch(), which trips the no-raw-channel-fetch boundary guard. Route it through postJsonRequest (the same guarded helper the AI-Studio route uses) so SSRF/dispatcher policy and timeout handling apply uniformly; drop the manual AbortController. Also allowlist the pre-existing Veo metadata-server fetch (video-generation-provider.ts:44, http://metadata.google.internal — link-local, must be raw; the SSRF guard intentionally blocks it). It predates this work and was surfaced when the PR first touched the package. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Transplants the still-needed /opt/openclaw local hot-patches onto fork-main so the membrane VM can cut over to the fork and ship native audio (#4). Veo (REST-bypass + 1080p), generation-provider-metadata, register.send — all upstream-untouched in 99d96c1→94db48d0, transplanted verbatim. session-lock openclaw#195 dropped (upstream no-op'd it). dispatch.ts (ADR 0015 inbound-sequencing) + dotenv deferred as fast-follow. Refs imperfect-co/tulgey#218.
Summary by CodeRabbit
Release Notes
New Features
--as-voiceflag to CLI send command for sending audio as voice notes.Updates
--force-documentflag documentation for Telegram-specific media handling.