Skip to content

feat(libtv-video): direct Feishu delivery via detached background waiter#973

Merged
alchemistklk merged 8 commits intomainfrom
feat/libtv-skill-requirements
Apr 10, 2026
Merged

feat(libtv-video): direct Feishu delivery via detached background waiter#973
alchemistklk merged 8 commits intomainfrom
feat/libtv-skill-requirements

Conversation

@alchemistklk
Copy link
Copy Markdown
Contributor

@alchemistklk alchemistklk commented Apr 9, 2026

What

Rework the libtv-video bundled skill so video results are delivered directly to the originating Feishu chat via a detached background waiter that calls the Feishu file/message APIs. Removes the earlier HTTP-notify controller route and the interim sessions_spawn/subagent attempt, keeping only the minimum stable routing state ({channel, chat_id}).

Why

Two prior attempts at libtv delivery were silently broken:

  1. POST /api/internal/libtv-notify (shipped in e7a18d86) routed through container.gatewayService.sendChannelMessage using persisted {channel, to, sessionKey, accountId, threadId, idempotencyKey}. account_id went stale across session migrations — the gateway couldn't resolve the account, and the user never received anything.
  2. sessions_spawn + subagent speech (interim refactor) relied on OpenClaw spawning a child session that inherited the parent's Feishu binding and then having the child's model speak the final message verbatim. Verified at the OpenClaw source level that the mechanism exists — but live testing showed two additional failure modes: the model can pick the wrong skill for video generation (tried liblib-ai-gen instead), and even when the right skill runs, there's no guarantee the child's model speaks the structured output verbatim.

What actually works is what Colin's manual Feishu probe proved: call /im/v1/messages with {app_id, app_secret} → tenant_access_token → open_id, using open_id from the inbound message metadata (stable per-user per-bot — never goes stale). This PR bakes that into the skill, with zero controller involvement.

How

Delivery chain:

create-session --channel feishu --chat-id ou_xxx
  └─ persists {channel, chat_id} in libtv-sessions.json
  └─ subprocess.Popen([wait-and-deliver, ...],
                      start_new_session=True,
                      stdin=DEVNULL, close_fds=True)
  └─ parent exits immediately with a submit-confirmation JSON
         ↓ (detached worker, survives parent exit)
  wait-and-deliver
  └─ polls libtv until terminal (tolerates transient RemoteDisconnected)
  └─ on success: shells out to feishu_send_video.py
       └─ downloads video → uploads via /im/v1/files → sends /im/v1/messages (msg_type: media)
       └─ if upload blocked by missing im:resource:upload scope,
          falls back to a text message with the video URL AND
          the grant URL (im:message scope is granted by default)
  └─ persists delivered_at idempotency marker

Key design decisions:

  • Detached worker, not subagent. sessions_spawn works but has too many moving parts (skill selection, model speech, OpenClaw hook inheritance). A plain Popen(..., start_new_session=True) is simpler and doesn't depend on the parent session staying alive or on any LLM speaking the result.
  • CLI args, not env vars. Contrary to the OPENCLAW_CHAT_ID convention that earlier libtv code assumed, OpenClaw does not inject inbound channel metadata as env vars into skill subprocesses. It passes inbound metadata to the model in the user message text. The model must extract it and pass it as --channel feishu --chat-id <ou_...> on every create-session invocation. SKILL.md documents this as a CRITICAL step with a concrete example. Same convention deploy-skill uses for its submit command.
  • Per-channel helper scripts. feishu_send_video.py lives next to libtv_video.py and is invoked via subprocess. Adding Slack/Discord/WeChat is just dropping a new <channel>_send_video.py helper and adding one branch in _deliver_results. No controller changes needed.
  • Permission-error fallback. If Feishu returns code: 99991672 (missing im:resource:upload), the helper parses the grant URL out of the error body and sends a plain text message containing the video URL + grant URL, so the user is still notified AND knows how to enable native-media delivery.
  • Idempotency via delivered_at. Re-invoking wait-and-deliver on an already-delivered session is a safe no-op.

Controller cleanup:

  • POST /api/internal/libtv-notify route + libtvNotifyRequestSchema + libtvNotifyResponseSchema + getBotIdFromSessionKey helper — deleted from apps/controller/src/routes/desktop-routes.ts.
  • apps/controller/tests/desktop-routes.test.ts — deleted (only contained libtv-notify tests).
  • tests/controller/desktop-routes.test.ts — deleted (re-export of above).
  • apps/controller/openapi.json, apps/web/lib/api/sdk.gen.ts, apps/web/lib/api/types.gen.ts — regenerated via pnpm generate-types.

Pre-merge code review (superpowers:code-reviewer) flagged two high-confidence issues before commit, both fixed:

  1. FD leak in _spawn_background_waiter — parent held the log file descriptor after Popen returned. Fixed by wrapping in try/finally and closing after Popen dup()s the fd into the child.
  2. Silent failure in upload_to_feishu when grant_url parsed but chat_id empty — now emits an explicit diagnostic referencing the missing context.

Affected areas

  • Desktop app (Electron shell) — bundled skill under apps/desktop/static/bundled-skills/libtv-video/
  • Controller (backend / API) — removed libtv-notify route + schemas + tests
  • Web dashboard (React UI) — regenerated SDK/types (no behavioral change)
  • OpenClaw runtime
  • Skills — libtv-video rewrite + new feishu_send_video.py helper
  • Shared schemas / packages
  • Build / CI / Tooling

Checklist

  • pnpm typecheck passes
  • pnpm lint passes
  • pnpm test passes (704 passed, 38 skipped — 6 are the rewritten libtv-video tests)
  • pnpm generate-types run (libtv-notify route removed from openapi.json + sdk.gen.ts + types.gen.ts)
  • No credentials or tokens in code or logs (reviewer verified app_secret and tenant_access_token are never printed to stderr or the waiter log)
  • No any types introduced

Notes for reviewers

End-to-end validation (real network calls):

  • CLI smoke test against the real libtv direct API (sk-libtv-... key): create-session → persisted → forked waiter → polled → terminal with a real .mp4 URL on libtv-res.liblib.art, auto-downloaded locally to ~/Downloads/libtv_results/.
  • Live Feishu DM test via hot-swapped state-dir skill on the running packaged Nexu.app (0.1.10):
    • Prompt: "请用 libtv-video 生成一个日出山景的 5 秒视频"
    • Session JSONL confirmed the model ran --help first to discover the new --channel/--chat-id args (proves the updated SKILL.md was loaded), then invoked create-session "..." --channel feishu --chat-id ou_33314772052f837a3cb2f919aa4605de.
    • Persisted libtv-sessions.json had delivery: {channel: "feishu", chat_id: "ou_333..."}, status: "completed", delivered_at: "2026-04-09T20:13:29.692667".
    • Waiter log (~/.nexu/libtv-waiter-<session-id>.log) captured the full chain: spawn → poll → terminal → download (3090KB) → upload to Feishu (file_key: file_v3_0010j_63f039c9-...) → send media message → ✅ Delivered to originating channel.
    • Native video message arrived in the Feishu chat. Total round-trip from submit to delivered: ~3 seconds.

Review focus areas:

  1. _spawn_background_waiter fd lifecycle (apps/desktop/static/bundled-skills/libtv-video/scripts/libtv_video.py) — reviewer flagged, fixed with try/finally.
  2. _extract_permission_grant_url regex and the permission-fallback path (feishu_send_video.py) — reviewer flagged the empty-chat_id branch, fixed with explicit diagnostic.
  3. SKILL.md "CRITICAL: always pass --channel and --chat-id" — this is the instruction the model has to obey every turn.

Out of scope / follow-ups:

  • Multi-channel support (Slack, Discord, WhatsApp, WeChat) — add <channel>_send_video.py helpers and branch in _deliver_results. The architecture already accommodates this.
  • scripts/probe/feishu-send-probe.mjs as a reusable "skill vs channel" bisection tool.
  • Group-chat delivery — currently the helper uses the open_id / chat_id prefix detection from medeo-video; group chats with thread-required replies would need additional handling.

@sentry
Copy link
Copy Markdown

sentry Bot commented Apr 9, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Replace the sessions_spawn + subagent delivery attempt (and the earlier
HTTP-notify callback path) with a simple, stable design: create-session
forks a detached wait-and-deliver process, and that process calls the
Feishu file/message APIs directly on terminal success.

Skill changes (apps/desktop/static/bundled-skills/libtv-video/):
- scripts/libtv_video.py:
  - create-session takes --channel and --chat-id CLI args. The model
    extracts sender_id from the inbound Feishu metadata block and
    passes them explicitly, the same way deploy-skill's submit command
    does.
  - Persist only {channel, chat_id} in libtv-sessions.json. No more
    account_id / session_key / thread_id — those were the root cause
    of the earlier silent-delivery bug where stale account_id values
    made the controller gateway unable to route.
  - _spawn_background_waiter uses subprocess.Popen(start_new_session=
    True, stdin=DEVNULL, close_fds=True) to fork a detached worker
    that survives the parent's exit. Parent closes its copy of the log
    file descriptor in a try/finally to avoid fd leaks.
  - cmd_wait_and_deliver polls libtv until terminal, then shells out
    to feishu_send_video.py for each result URL. Persists a
    delivered_at idempotency marker so re-invocations on a delivered
    session are a safe no-op.
  - Transient network errors (RemoteDisconnected, ConnectionError,
    URLError, TimeoutError) in the poll loop are caught and retried
    on the next tick instead of crashing the waiter.

- scripts/feishu_send_video.py (new file):
  - Copied from medeo-video's proven helper as the starting point.
  - Extended with _extract_permission_grant_url +
    _handle_upload_permission_error: if upload fails with Feishu's
    im:resource:upload scope error (code 99991672), fall back to a
    plain text message containing the video URL AND the grant URL
    so the user can enable native-media delivery for future runs.
    Uses the already-granted im:message scope.
  - On permission error with no chat_id available, emit an explicit
    diagnostic instead of silently falling through.

- SKILL.md:
  - New delivery contract section explaining the background waiter
    pattern.
  - CRITICAL instruction that the model must extract sender_id from
    the inbound Feishu metadata block and pass it as --chat-id on
    every create-session invocation. Includes a concrete example.
  - Updated guard checklist to match the new contract.

Controller cleanup (apps/controller/):
- Delete POST /api/internal/libtv-notify, its request/response
  schemas, and the getBotIdFromSessionKey helper from
  src/routes/desktop-routes.ts.
- Delete the whole apps/controller/tests/desktop-routes.test.ts file —
  it only contained tests for the removed route.
- Delete tests/controller/desktop-routes.test.ts (re-export of above).
- Regenerate openapi.json, sdk.gen.ts, types.gen.ts via
  pnpm generate-types.

Tests (tests/desktop/libtv-video-skill.test.ts):
- Full rewrite against the new contract. 6 tests cover: Seedance URL
  constant, submit-confirmation JSON shape, persisted delivery block,
  empty-delivery fallback, explicit video ratio override, sk-libtv-
  direct API routing, malformed submit rejection. All green.
- Use LIBTV_SKIP_BACKGROUND_WAITER=1 to prevent tests from leaking
  detached processes into CI.

E2E validation:
- CLI smoke-tested against real libtv direct API with sk-libtv- key —
  real video generated and locally downloaded.
- Live Feishu DM test with hot-swapped state-dir skill: the model
  read the updated SKILL.md, ran create-session with --channel feishu
  --chat-id ou_333..., the detached waiter polled libtv, invoked
  feishu_send_video.py, and delivered a native media video message
  to the user's Feishu chat. End-to-end round trip: ~3 seconds after
  create-session.

Follow-ups (not in this commit):
- Support additional channels by dropping a <channel>_send_video.py
  helper alongside feishu_send_video.py and adding a branch to
  _deliver_results.
- Add a scripts/probe/feishu-send-probe.mjs as a reusable smoke probe
  for bisecting "skill vs channel" delivery failures.
@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Apr 9, 2026

Deploying nexu-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 7ed36f2
Status: ✅  Deploy successful!
Preview URL: https://c2229b49.nexu-docs.pages.dev
Branch Preview URL: https://feat-libtv-skill-requirement.nexu-docs.pages.dev

View logs

@alchemistklk alchemistklk changed the title feat: harden libtv delivery notifications feat(libtv-video): direct Feishu delivery via detached background waiter Apr 9, 2026
Rename the bundled skill's display name from "LibTV Video" to
"LibTV - Image&Video(Seedance 2.0)" and update the description to
make it clear the skill supports BOTH image and video generation,
powered by Seedance 2.0.

The skill directory (libtv-video) and ledger slug stay the same so
persisted state, the compiled openclaw config, and the e2e delivery
path from the previous commit remain valid.

Updated surfaces:
- SKILL.md frontmatter name + description
- SKILL.md H1 heading
- libtv_video.py module docstring
- libtv_video.py ArgumentParser description
- Rewrite SKILL.md frontmatter description to match the approved copy:
  "Seedance 2.0 video & image generation via LibTV Gateway - AI
  text-to-video, image-to-video, video continuation, style transfer,
  and text-to-image using Seedance 2.0 model. Also supports Kling 3.0,
  Wan 2.6, Midjourney, Seedream 5.0. Trigger phrases: ..."
  This expands trigger coverage to image-generation verbs (draw,
  generate image, make an image) and enumerates the supported
  alternative models (Kling 3.0, Wan 2.6, Midjourney, Seedream 5.0).

- Drop the hardcoded MODEL_HINT constant and _append_model_hint helper
  from libtv_video.py. The skill used to silently append
  ", please use Seedance 2.0" to every prompt that did not mention a
  model, which forced Seedance 2.0 even when the user was happy with
  the LibTV backend default or wanted another supported model.
  _build_session_message now only appends the video ratio hint.
Add replaceLibtvVideoFromBundle and call it from
skillhub-service.initialize() after copyStaticSkills. Unconditionally
wipes and re-copies the bundled libtv-video into the state dir on
every controller startup, then upserts the managed ledger record to
status: installed. Ships bundled libtv-video updates (including the
detached-waiter + direct-Feishu-delivery refactor) to existing users
on their next app boot.

Bypasses copyStaticSkills on purpose — its knownSlugs check skips on
any ledger source, which would silently fail if the user has a
workspace or user-level libtv-video entry. The dedicated function
only touches the state-dir copy under <openclawSkillsDir>/libtv-video/
and only the managed ledger record. User-scoped copies in
~/.agents/skills/ and per-agent copies under agents/<id>/skills/
are left untouched. Does not respect the managed record's
uninstalled status — libtv-video is treated as a core bundled
capability that always tracks the shipped version.

New tests in apps/controller/tests/replace-libtv-video-from-bundle.
test.ts (5 cases, real SkillDb on a temp ledger + real filesystem):
- fresh-install
- replaces stale content while keeping managed record installed
- resurrects over an uninstalled managed record instead of honoring it
- leaves workspace records for the same slug byte-identical
- returns bundle-missing when bundled source dir is absent

Updated the existing skillhub-service test mock to expose the new
export so the 18 mocked-SkillDb cases still pass.
@alchemistklk alchemistklk marked this pull request as ready for review April 9, 2026 13:54
docs/superpowers/ is a working directory for agent-generated plans and
design notes that should not be tracked in the repo. Add it to
.gitignore. Remove the stale direct-key routing design doc committed
in adf2ece — it describes an interim approach that has since been
superseded by the detached background waiter + direct Feishu delivery
design landed in 5fca9ad.
@alchemistklk
Copy link
Copy Markdown
Contributor Author

/cr

@slack-code-review-channel
Copy link
Copy Markdown

✅ CR topic created in Feishu topic group Refly CR.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: de86300613

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread apps/desktop/static/bundled-skills/libtv-video/scripts/feishu_send_video.py Outdated
main() was ignoring the False return from send_video_message and
exiting 0, which caused _deliver_feishu_video in libtv_video.py to
treat the failed send as successful and set delivered_at — the user
would never receive the video but the skill would record delivery as
complete. Now sys.exit(1) on False so the caller correctly detects the
failure and skips setting delivered_at.
@alchemistklk alchemistklk merged commit 01ac8ea into main Apr 10, 2026
16 of 17 checks passed
@lefarcen lefarcen mentioned this pull request Apr 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants