Skip to content

Stream subscriptions carry state; the browser itx layer becomes one hook (useItx)#1472

Merged
jonastemplestein merged 5 commits into
mainfrom
stream-subscribe-carries-state
Jun 10, 2026
Merged

Stream subscriptions carry state; the browser itx layer becomes one hook (useItx)#1472
jonastemplestein merged 5 commits into
mainfrom
stream-subscribe-carries-state

Conversation

@jonastemplestein

@jonastemplestein jonastemplestein commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

What

Two halves of one decision: stream subscriptions become the ONE reactive primitive, and the browser layer collapses onto it — one hook, no query cache, no SSR, no reconnect machinery. Net −951 lines against main, with the deleted test files making the suite smaller than it was.

Half 1 — the protocol: subscriptions carry reduced state (DECISIONS D20)

  • Every batch carries state. The pump (now in CoreStreamProcessor.openConnection after the presence-roster merge) attaches the core reduced state to every delivery, read in the same synchronous block as streamMaxOffset so the two always correspond. The OS capability projects it through the exact mapping getState() uses (coreStateToStreamState is the single shared projection), so subscribe-state === getState-state by construction.
  • events: false = state-only mode. Same batches with events: [] (empty array, not omitted — shape stability), one delivery per state advance with missed appends coalesced. Replay without events is meaningless, so state-only subscriptions are implicitly live-from-now.
  • The initial push. EVERY subscription immediately receives one batch (current state + streamMaxOffset, folded into the replay batch when there is one), so a subscriber paints its first render from the subscription alone — no separate getState call.
  • Kernel surface. ItxStream.subscribe(onBatch, { afterOffset, events? }) plus ItxStream.onStateChange(cb) sugar (state-only subscribe with explicit callback-stub retention).

Half 2 — the browser layer: ONE hook (DECISIONS D21, supersedes D19 and one-socket-per-tab)

The TanStack-Query-for-itx + SSR + shared-socket architecture was too complicated. apps/os/src/itx/use-itx.ts replaces all of it:

  • useItx(context?) suspends (React 19 use() on a per-context module-singleton promise) until a WebSocket to /api/itx[/<ctx>] is connected, and returns the live RpcStub<Itx>. getBrowserItx(context?) is the same singleton for non-hook code. That's the whole API.
  • No reconnect machinery. Socket death evicts the map entry; subscribers re-render via useSyncExternalStore, dial fresh, re-suspend — the initial push repaints them with current state. No backoff, no offset resume, no liveness probes. Multiple sockets per tab are fine.
  • No SSR for itx components — the hook THROWS on the server. A forever-pending use() during streaming SSR would hold the response stream open (React waits for suspended boundaries); a throw inside a Suspense boundary streams the fallback and recovers client-side, and outside one it fails loudly instead of hanging the worker. Consumers sit under ssr: false routes (streams pages), <ClientOnly> (the repl's activity tail), or the admin layout's client-only connect gate.

Consumers rewritten: the stream tree browser is now LIVE (one onStateChange per loaded node, refresh = re-subscribe; react-query gone), the streams index/detail routes use useItx under Suspense (loader prefetch deleted), breadcrumb popovers fetch childPaths on open into local state, and ItxActivityTail is a kernel subscribe from "start" into component state (500-event cap, offset-deduped re-subscribe). The repl keeps its own createBrowserReplSession — it needs dispose/reconnect-on-demand semantics the singleton deliberately lacks.

Deleted

apps/os/src/itx/react/ (provider, backoff-reconnect connection, query bridge, stream-tail multiplexer, useStreamEvents, all their tests), itx/server.ts (getServerItx), itx/loader.ts (getLoaderItx/prefetchItxQuery), lib/itx-queries.ts, and the itx-server-handle worker test harness (+ its package.json script and knip entries). itx/errors.ts stays (error codes still matter to catch blocks); its test moved next to it.

Docs

DECISIONS D20 + D21, itx-orpc-replacement-plan.md browser/SSR sections rewritten to the one-hook model, tasks/os-orpc-teardown.md updated (getServerItx marked superseded; route-conversion guidance now references useItx).

Notes

  • Merged with main's subscriber-presence work (streams: subscriber presence facts + reconciler homogenization #1460): the D20 pump changes were re-applied inside CoreStreamProcessor.openConnection, and subscribe() carries both new options (events, subscriber). Tests that asserted exact batch counts/empty-events were reconciled with presence facts (every subscribe/unsubscribe appends one) to assert delivered content and per-batch invariants instead.
  • subscribeOutbound (processor hosts) shares the pump, so outbound connections also get state-bearing batches and an initial events: [] batch on (re)connect — StreamProcessor.ingest no-ops on empty batches.

Tests

  • Pump level (packages/streams stream.workers.test.ts): state matches the DO's own reduced state on every batch; immediate initial batch on live-only subscribe; initial push folds into replay; events: false state-only mode — all green alongside main's presence-roster tests.
  • Capability loopback (pnpm test:itx-stream-subscribe, 13 passing): batch state === getState shape end-to-end, live-only initial push, state-only mode, onStateChange across real Workers RPC hops.
  • Hook core (apps/os/src/itx/use-itx.test.ts): the connection map — stable entry per context, eviction + notification on death, identity-guarded eviction, fresh dial after eviction.
  • e2e (itx-subscribe.e2e.test.ts, runs in preview CI) unchanged.
  • Full: pnpm typecheck && pnpm lint && pnpm knip && pnpm format && pnpm test all green at root after merging main.

🤖 Generated with Claude Code


Note

Medium Risk
Large architectural cutover across streams protocol, OS UI, and deleted SSR/prefetch paths; live views may stall until refresh if Stream DOs evict without server-side subscription recovery.

Overview
Stream subscriptions now carry reduced state on every batch (same shape as getState()), with an immediate initial push so UIs can paint without a separate fetch. events: false is state-only mode (live-from-now, coalesced updates). ItxStream.onStateChange is the reactive sugar on top.

Browser itx collapses to useItx / getBrowserItx: per-context WebSocket singletons, Suspense until connected, no SSR (throws on server), no TanStack Query bridge, reconnect backoff, stream-tail multiplexer, or getServerItx loader prefetch. ItxProvider and ~/itx/react/ are removed.

UI is rewired: the stream tree uses live onStateChange per node; streams routes are ssr: false + Suspense; breadcrumbs getState on popover open; ItxActivityTail uses kernel subscribe from "start". Docs/decisions (D20, D21) and tests are updated; the itx-server-handle harness is deleted.

Reviewed by Cursor Bugbot for commit 3d7754a. Bugbot is set up for automated code reviews on this repo. Configure here.

Environment Config Lease

No active environment config lease.

OS

Status: released
Commit: 3d7754a
Preview: https://os.iterate-preview-6.com
Summary: Preview app released.
Workflow run
Updated: 2026-06-10T20:23:23.429Z

jonastemplestein and others added 3 commits June 10, 2026 20:23
The stream subscription protocol now serves both events and reduced state:

- Every delivered batch carries `state` — the stream's core reduced state as
  of `streamMaxOffset`, read in the same synchronous block so the two always
  correspond. The OS capability projects it through the exact getState
  mapping (coreStateToStreamState), so subscribe-state === getState-state.
- `events: false` is state-only mode: same batches with `events: []`, one
  delivery per state advance (missed appends coalesce), implicitly
  live-from-now since replay without events is meaningless.
- The initial push: EVERY subscription immediately receives one batch
  (current state + streamMaxOffset + any replayed events), so a subscriber
  paints its first render without a separate getState call. Live-only
  subscriptions no longer stay silent until the next append.
- Kernel surface: ItxStream.subscribe batches gain `state`; new
  ItxStream.onStateChange(cb) sugar = subscribe(b => cb(b.state),
  { events: false, afterOffset: "end" }), with callback-stub retention since
  the sugar's wrapper outlives the originating RPC.

Recorded as itx DECISIONS D20; supersedes getState-polling reactivity and
serves the upcoming one-hook browser layer (useItx + onStateChange).

Tested at the pump (stream.workers.test.ts), through the capability loopback
(test:itx-stream-subscribe), and over capnweb (itx-subscribe.e2e.test.ts).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The TanStack-Query-for-itx + SSR + shared-socket architecture is gone.
The browser layer is now a single file, src/itx/use-itx.ts:

- useItx(context?) suspends (React 19 use()) until a per-context
  module-singleton WebSocket to /api/itx[/<ctx>] is connected and
  returns the RpcStub<Itx>. getBrowserItx(context?) is the same
  singleton for non-hook code.
- No query cache, no refcounting, no reconnect machinery, no
  multiplexer, no SSR. Socket death evicts the map entry; subscribers
  re-render, dial fresh, re-suspend — the kernel's initial state push
  (D20) repaints them. Multiple sockets per tab are fine.
- The hook THROWS during SSR (a forever-pending use() would hold the
  streaming response open); consumers sit under ssr:false routes,
  <ClientOnly>, or the admin layout's client-only connect gate.

Consumers rewritten:
- stream-tree-browser: react-query out, live onStateChange per loaded
  node (the tree updates as events land); refresh = re-subscribe.
  Source is now just path -> stream handle, shared by the project and
  admin explorers.
- streams index/detail routes: useItx under a Suspense boundary,
  loader prefetch deleted, index goes ssr:false.
- path-breadcrumbs: sibling/child popovers fetch childPaths on open
  via getBrowserItx into local state.
- itx-activity-tail: kernel subscribe from "start" into component
  state, 500-event cap, offset-deduped re-subscribe.
- _app.tsx loses ItxProvider; the repl keeps its own session by design.

Deleted: src/itx/react/* (connection, provider, hooks, stream-tail
multiplexer, useStreamEvents + their tests), itx/server.ts
(getServerItx), itx/loader.ts (getLoaderItx/prefetchItxQuery),
lib/itx-queries.ts, and the itx-server-handle worker test harness.
errors.test.ts moved next to the kernel module it tests.

Docs: DECISIONS D21 (supersedes D19 and one-socket-per-tab),
itx-orpc-replacement-plan browser/SSR sections rewritten,
os-orpc-teardown task updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The presence-roster work (#1460) moved the delivery pump into
CoreStreamProcessor.openConnection; re-applied the D20 protocol there:
batches carry state read alongside streamMaxOffset, every subscription
gets an initial push, events: false is state-only mode. subscribe()
keeps both new options (events + subscriber) end to end, including the
capnweb RpcTarget override.

Tests reconciled with presence facts (every subscribe/unsubscribe now
appends a fact, so exact batch counts/empty-events assertions raced):
assert on delivered content and per-batch invariants instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jonastemplestein jonastemplestein changed the title Stream subscriptions carry reduced state: one reactive primitive Stream subscriptions carry state; the browser itx layer becomes one hook (useItx) Jun 10, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7a7d6ac. Configure here.

Comment thread apps/os/src/components/path-breadcrumbs.tsx
Comment thread apps/os/src/components/stream-tree-browser.tsx
Bugbot on #1472: a reopened breadcrumb popover (or stream navigation)
painted the previous stream's siblings until the new fetch landed, and
the tree's refresh() kept showing the stale live state while the new
subscription connected — defeating its purpose as the recovery action
for a silently stalled subscription. Both hooks now reset to their
loading state whenever their effect re-runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jonastemplestein jonastemplestein merged commit 082ad73 into main Jun 10, 2026
11 checks passed
@jonastemplestein jonastemplestein deleted the stream-subscribe-carries-state branch June 10, 2026 20:21
jonastemplestein added a commit that referenced this pull request Jun 10, 2026
… legacy define compat, nested workspace.git (#1476)

The parked debts from itx-next.md, bandaid pulled to the max per review
— **no backcompat anywhere** (prd/preview/dev itx data is erasable).

## CodemodeSession tombstone — deleted

The tombstone DO class, its `codemode-session-local` namespace, the
`CODEMODE_SESSION` binding, and the vitest wrangler entries are gone.
Alchemy emits the `deleted_classes` migration (mechanism proven in
#1464). Streams that still carry durable subscriber events dialing the
namespace will error on delivery — accepted, the data is being erased.

## executeCodemodeFunctionCall — protocol fully deleted (~2.5k lines)

Every capability entrypoint loses its legacy dispatch method (agents,
gmail, repos, secrets, slack, streams, workspace, openapi-bridge,
AiCapability, OrpcCapability, test entries), along with
`legacy-codemode-call.ts` and each wrapper's dead arg-validation
helpers. Two callers were still alive and got clean replacements:
- the agent chat/debug tool path now dials
`AgentDurableObject.callAgentTool({ tool, path, args, callId })`
- the ingress test entry upserts secrets via `OrpcCapability.call({
path: ["secrets", "upsert"], … })`

`packages/shared/src/codemode/` → `packages/shared/src/type-tree/`
(context-proxy deleted — only importer was a deleted legacy test
provider); generated typing identifiers de-codemoded (`ItxConsole`,
`generateContextTypesFromJsonSchema`, …).

## Registry legacy compat — deleted

`caps.define` takes a `target` (SerializableCapTarget) **only**, end to
end (registry, ContextDO, Project DO, handle, REPL typings): the legacy
`source`/`kind: "worker" | "facet"` inputs, the `codeId` spelling of
`cacheKey`, stored `worker`/`facet` kinds, the `source_json` rollback
column + sync writes, and all read-side normalization are gone. All
callers (browser REPL examples, every e2e suite) spell rpc/source
targets directly.

## Nested `itx.workspace.git.*` — deleted

Nested RpcTargets returned from entrypoint getters don't survive RPC
boundaries; the flat
`gitClone`/`gitAdd`/`gitCommit`/`gitPush`/`gitStatus` methods are the
surface. The agent preset prompt was actively teaching the broken nested
spelling — fixed.

## allowedHosts debt — closed as misdiagnosis

The config is `allowedHosts: true`; the 403/502s came from a wedged vite
process behind the still-connected cloudflared tunnel. Documented in
itx-next.md; e2e through `os.iterate-dev-jonas.com` verified passing.

## Verification

`pnpm typecheck` / `lint` / `knip` / `format` green; apps/os unit tests
green; workerd suites (`test:project-ingress` 6/6,
`test:itx-server-handle` 5/5 pre-#1472, `test:type-tree` 113/113) green;
itx e2e (itx, fork, http, subscribe — 20 tests) green against a local
dev server on the merged tree, including the post-#1474 capnweb fork.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **High Risk**
> Large breaking change to capability registration, MCP/ingress
execution paths, and production streams that may still dial removed
CodemodeSession subscribers; preview/dev data is intentionally erasable.
> 
> **Overview**
> This PR **removes the last codemode-era wiring** and tightens the itx
capability model with **no read-side backcompat**.
> 
> **CodemodeSession** is fully removed: the tombstone Durable Object,
`CODEMODE_SESSION` binding/namespace in Alchemy and vitest wrangler
configs, worker exports, and the dedicated test script. MCP and tests
now assume execution goes through the shared itx runner.
> 
> **Legacy `executeCodemodeFunctionCall` dispatch is deleted** across
capability entrypoints (~2.5k lines), along with
`legacy-codemode-call.ts` and shared `codemode/context-proxy`. Call
sites move to itx-native paths (`callAgentTool`, `OrpcCapability.call`,
OpenAPI `call({ path, args })`). **`packages/shared` codemode** is
renamed to **`type-tree`** with de-codemoded type-generation names.
> 
> **`caps.define` is target-only**: required `SerializableCapTarget`, no
`source`/`kind`/`codeId`, no `source_json` column or
`normalizeCapTarget` on read—stored rows use `target_json` verbatim.
Docs, REPL typings, e2e, and browser REPL examples all use `{ type:
"rpc", worker: { type: "source", source: { cacheKey, … } } }`.
> 
> **Workspace git** drops the nested `git` RpcTarget getter; agents,
presets, e2e, and preview scripts use flat **`gitClone` / `gitAdd` /
`gitCommit` / `gitPush` / `gitStatus`**. **itx-next.md** records
resolved debts (legacy define, nested git, allowedHosts misdiagnosis).
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
9cf0edd. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

<!-- CLOUDFLARE_PREVIEW -->
## Environment Config Lease
<!-- CLOUDFLARE_PREVIEW_STATE -->
<!--
{
  "apps": {
    "os": {
      "appDisplayName": "OS",
      "appSlug": "os",
      "status": "deployed",
      "updatedAt": "2026-06-10T20:40:22.067Z",
      "headSha": "9cf0eddaebfbcf1516be75f3e6acffdc31004f08",
      "message": null,
      "publicUrl": "https://os.iterate-preview-6.com",
"runUrl": "https://github.com/iterate/iterate/actions/runs/27304663825",
      "shortSha": "9cf0edd"
    },
    "semaphore": {
      "appDisplayName": "Semaphore",
      "appSlug": "semaphore",
      "status": "deployed",
      "updatedAt": "2026-06-10T20:31:34.680Z",
      "headSha": "0f50df98f943df63d8133e6be05dcdb33e77305c",
      "message": null,
      "publicUrl": "https://semaphore.iterate-preview-6.com",
"runUrl": "https://github.com/iterate/iterate/actions/runs/27304258909",
      "shortSha": "0f50df9"
    }
  },
  "environmentConfigLease": {
    "dopplerConfig": "preview_6",
    "leasedUntil": 1781127403360,
    "leaseId": "6fe10f72-e7bb-431a-9402-1abae9841129",
    "slug": "preview-6",
    "type": "environment-config-lease"
  }
}
-->
<!-- /CLOUDFLARE_PREVIEW_STATE -->
Lease: `preview-6`
Doppler config: `preview_6`
Type: `environment-config-lease`
Leased until: 2026-06-10T21:36:43.360Z

### OS
Status: deployed
Commit: `9cf0edd`
Preview: https://os.iterate-preview-6.com
[Workflow
run](https://github.com/iterate/iterate/actions/runs/27304663825)
Updated: 2026-06-10T20:40:22.067Z

### Semaphore
Status: deployed
Commit: `0f50df9`
Preview: https://semaphore.iterate-preview-6.com
[Workflow
run](https://github.com/iterate/iterate/actions/runs/27304258909)
Updated: 2026-06-10T20:31:34.680Z
<!-- /CLOUDFLARE_PREVIEW -->

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant