Skip to content

Guidance for streaming chat responses in DevTools integrations #306

@onmax

Description

@onmax

Clear and concise description of the problem

I’m working on improving the DX of Chat SDK usage in Nitro v3 and want to add a DevTools page for inspecting and interacting with chats.

A key use case is streaming assistant deltas into the DevTools iframe so chat responses render incrementally. This can be built today with an action call plus delta/done/error events, but then every integration has to invent its own stream ids, cancellation, error handling, and recovery conventions.

I’m open to contributing the necessary PRs in DevTools/devframe or birpc, depending on the preferred direction.

Suggested solution

I’d like guidance on the recommended pattern:

  1. Document action + events as the preferred approach.
  2. Add a small DevTools/devframe helper that exposes an AsyncIterable-style API over action + events.
  3. Explore explicit streaming support in birpc, such as $stream() / asStream().

Alternative

Each integration can keep implementing its own action + event streaming protocol.

Additional context

Related birpc discussion: antfu-collective/birpc#20. I understand the concerns about cancellation, disconnects, timeouts, and keeping birpc small.

Validations

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions