itx follow-ups: intercept tunnel deleted, streams is a cap, scoped DO names, REPL reads types.ts by jonastemplestein · Pull Request #1490 · iterate/iterate

jonastemplestein · 2026-06-11T07:02:12Z

The four follow-ups the consolidation arc left written down in itx-next.md, in one PR. Net −578 lines while adding a whole new capability surface. No backcompat anywhere.

1. The egress intercept tunnel is deleted — shadowing was already the feature

Motivation. #1487 made fetch an ordinary shadowable capability. At that point the captun-based intercept tunnel was a second, hand-built implementation of the same idea: "while I'm connected, route the project's egress through me." It had its own DO field, accept route, keepalive plumbing, a special projectEgressInterceptActive flag threaded through secret substitution, a benchmark script, and a bespoke test harness. All of it is expressible as one caps.define.

Before (the tunnel era):

// Client: dial a special captun endpoint on the project's ingress
const tunnel = await createCaptunTunnel({
  url: `${ingressUrl}/__iterate/intercept-project-egress`,
  headers: { Authorization: `Bearer ${adminToken}` },
  fetch: myFetch,
});
// Server: #projectEgressInterceptTunnel field, accept/replace/teardown
// lifecycle, an egressFetch branch, and a substitution mode that withheld
// real secret material while a tunnel was connected.

After (it's just a cap):

using itx = connectItx({ baseUrl, token, context: projectId });
class Interceptor extends RpcTarget {
  async call({ args }) { return await myFetch(args[0]); }  // args[0] is the Request
}
await itx.caps.define({ name: "fetch", invoke: "path-call", target: new Interceptor() });
// ALL project egress — itx.fetch() and bare fetch() in every loaded isolate —
// now flows through myFetch. Drop the session and the default pipe resurfaces.

The security property came along for free and got simpler: the tunnel needed an explicit "withhold secrets while intercepted" mode inside substitution; a shadow provider simply never reaches the substituting pipe, so it sees getSecret(...) placeholders verbatim. The flag is gone; substitution always yields real material on the one path that has any.

The e2e fixture's egressFetch option keeps its exact API (now implemented as above), and the workerd ingress test was rewritten to the cap story — which caught a real bug: probing onRpcBroken on a Workers-RPC live provider rejects unhandled (jsrpc proxies every property as a remote method); the registry now treats it as best-effort.

2. `streams` joins the platform defaults

Motivation. After #1482/#1487 the kernel was caps, streams, fork, project, projects, describe. streams was only still hardwired because its access checks live in the handle. But split the concern in two and the blocker dissolves: on a project context the namespace is forced to the project (there is no access decision to make — registry-injected projectId props pin it, definers can't point it elsewhere), and only the global namespace genuinely needs the connect-time access set.

Before: ItxStreams/ItxStream hardwired into handle.ts, reserved name, not shadowable.

After:

// platform:project (code-contexts.ts) — just another definition:
caps.define({
  name: "streams",
  target: { type: "rpc", worker: { type: "loopback" }, entrypoint: "StreamsCap" },
});

// …which means a context can now shadow its own event-stream surface:
await itx.caps.define({ name: "streams", invoke: "path-call", target: myStreamsFake });

The collection/stream classes moved to src/itx/caps/streams.ts, parameterized by an explicit StreamsScope { access, exports }; the handle's getter branches — project handles resolve through the registry (shadowable), global handles keep the kernel branch for the deployment-wide "global" namespace gated on access === "all". Everything else is unchanged: absolute refs ("ns:/path") still go through the one access check with NOT_FOUND masking.

Two things made this safe to ship rather than scary:

Chained calls ride RPC promise pipelining. itx.streams.get("/x").append(e) crosses a boundary in every real execution mode (capnweb from browsers/Node, jsrpc from loaded isolates), and both transports pipeline follow-up calls onto returned RpcTargets — the same shape itx.agents.create().doThing() already proved.
Subscriptions survive the extra hop. subscribe callbacks now cross client → registry DO → StreamsCap → Stream DO; the existing dup-discipline in StreamsCapability holds, proven by the subscribe e2e suite running unchanged.

3. Durable-object dials are name-scoped

Motivation. #1482 shipped { type: "durable-object", binding, name } refs behind an empty allowlist, because raw names meant an allowlisted namespace would let any project dial any other project's instances — documented as "the open design before any namespace can join."

Resolved: the registry now dials

namespace.getByName(`itx:${projectId}:${name}`)

so every allowlisted namespace's itx-reachable instances are disjoint per project by construction — the definer's name is a label inside their project's slice, not a global address. Deployments can now actually use APP_CONFIG_ITX.dialableDurableObjects for namespaces designed for itx use. (Namespaces whose existing instances matter — PROJECT, STREAM — still don't belong on the list: itx dials would reach fresh, empty objects, which is exactly the point.)

4. The REPL editor consumes types.ts — drift is now structurally impossible

Motivation. apps/os/src/itx/types.ts is the handwritten design-of-record for the whole itx surface; the browser REPL's editor carried a second, hand-maintained 353-line ambient declaration with a "keep in sync" comment — the kind that's wrong within a week.

After: the REPL's TypeScript virtual FS loads types.ts verbatim (import source from "~/itx/types.ts?raw", which works in both the vite-bundled worker and vitest). The hand-written file shrank to a 120-line prelude declaring only what types.ts can't know: the session globals (itx, vars, projectId, RpcTarget, $_). A bonus improvement fell out: the capability fallthrough is now declared on the official KnownCaps merge point, so handles returned by itx.fork() / itx.projects.get() carry it too — (await itx.projects.get(id)).slack.chat.postMessage typechecks in the editor, which the old ambient got wrong. Tests assert the editor sees types.ts-only markers, so regressions are loud.

The kernel after this PR

caps, fork, project, projects, describe        ← the trust kernel
streams (global namespace only)                ← connect-time access, by nature
─────────────────────────────────────────────
ai, fetch, streams, repos, workspace, worker   ← platform:project definitions,
                                                  i.e. literally the data
                                                  structures caps.define takes,
                                                  every one of them shadowable

Verification

pnpm typecheck / lint / knip / format green; apps/os unit tests 222 green; workerd test:project-ingress 6/6 (including the rewritten fetch-shadow-sees-placeholders test); itx e2e — 32 tests across core/fork/http/subscribe suites, exercising streams through the registry path, the example catalogue in every runtime, fork workspace isolation, and the full fetch-shadow story — green against a local dev server.

🤖 Generated with Claude Code

Note

Medium Risk
Touches project egress, secret substitution, and ingress/DO routing paths used in production and e2e; behavior changes are intentional but need regression on fetch shadowing and streams via the registry.

Overview
Removes the Project Egress Intercept Tunnel (captun route on the Project DO/ingress, tunnel state, intercept-specific secret withholding, benchmark script, and e2e captun helper) and replaces interception with a session-bound live fetch capability shadow on the project itx context—interceptors see getSecret(...) placeholders because substitution only runs on the default egress pipe.

streams becomes a shadowable platform:project default (StreamsCap loopback in caps/streams.ts); project handles resolve itx.streams through the registry while the global "global" namespace stays a kernel branch gated on admin access. Durable-object capability dials now use itx:<projectId>:<name> so allowlisted namespaces are per-project disjoint.

The browser REPL editor loads ~/itx/types.ts verbatim via ?raw instead of a large hand-maintained ambient file; docs/ADR/context and tests are updated accordingly, including registry best-effort onRpcBroken for Workers-RPC live providers.

^{Reviewed by Cursor Bugbot for commit 93efa03. Bugbot is set up for automated code reviews on this repo. Configure here.}

Environment Config Lease

No active environment config lease.

OS

Status: released
Commit: 93efa03
Preview: https://os.iterate-preview-2.com
Summary: Preview app released.
Workflow run
Updated: 2026-06-11T07:09:58.416Z

…PL reads types.ts The four follow-ups the consolidation pass left on the table: - The captun egress intercept tunnel is DELETED (-650 LOC): fetch-cap shadowing IS the intercept mechanism. The e2e fixture's egressFetch option survives, reimplemented as a live fetch cap on a dedicated itx session; the workerd ingress test was rewritten to the cap story and caught a real bug (onRpcBroken probing rejects unhandled on jsrpc live providers — now best-effort). Substitution always yields real material (the intercept-active withholding flag is gone; a shadow simply never triggers substitution). - streams joined the platform defaults: StreamsCap loopback, pinned to the owning project's namespace by registry-injected props; the collection/stream classes moved to itx/caps/streams.ts. The GLOBAL namespace stays kernel (gated on the connect-time access set, which no cap definition can express). Chained calls ride RPC promise pipelining; subscribe through the extra registry hop is e2e-proven. - durable-object dials are name-scoped (itx:<projectId>:<name>) so an allowlisted namespace's itx-reachable instances are per-project disjoint by construction — the open design blocking allowlisting. - The REPL editor consumes src/itx/types.ts verbatim (?raw into the TS virtual FS); itx-repl-types.ts shrank 353→120 lines to a prelude of session globals, and narrowed handles gained the cap fallthrough. Kernel after this PR: caps, fork, project, projects, describe, plus the global streams namespace. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…remove Merge resolution: drop `streams` from ITX_BUILTIN_NAMES (now a shadowable cap) keeping the typed facades; drop the unused getStreamsCapability and captun imports. projects.remove now delegates to ProjectsCapability.remove like create — enforcing per-project access (a non-admin can remove a project they can reach, not operator-only) and returning idempotent { deleted } by re-reading the row. Drops the dead requireAllAccess + now-unused imports. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ve mirrors create - Merge resolution: take main's fetch-cap egress shadow (captun intercept tunnel deleted), drop the deleted egress-tunnel benchmark. - projects.remove delegates to ProjectsCapability.remove without the (now removed) rethrow wrapper, matching create. - Preview smoke: create-or-find tries create then resolves by slug on CONFLICT (no list pagination → finds projects beyond page 1); admin secret reader also accepts OS_E2E_ADMIN_API_SECRET like os-client. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…types) Main's follow-ups PR independently shipped this branch's tunnel deletion — its versions win wholesale where they overlap: the e2e fixture's defineLiveEgressFetchCap, the one-shot define→fetch→revoke workerd shadow test, the registry's best-effort onRpcBroken wiring, and the withheld-text removal. Re-applied this branch's remaining unique layers on top: the DO keeps NO egress surface (main still had fetch/egressFetch as the terminal pipe; here the default `fetch` target is the stateless EgressPipe), wireIsolateEnv in the registry's loadWorker, auth-routed id minting in ItxProjects.create, and the ProjectCapability deletion (main's test entry still exported it). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…tercept dies, kernel shrinks, auth mints, legacy afterAppend deleted (#1485) ## What The remaining grand-cleanup workstreams in one deliberately breaking PR (prd gets redeployed). DECISIONS **D23** is the canonical record. Three main-side PRs landed mid-flight and overlap this work — all adopted wholesale in the merges: **#1482** (repos/workspace/worker as platform defaults with origin-carrying delegation), **#1487** (`fetch` is a shadowable cap, `define` absorbs `provide`, shared registry host), and **#1490** (intercept tunnel deleted, streams is a cap, best-effort `onRpcBroken`). This PR contributes the layers below on top of them. ### §9 finished: the egress pipe is stateless - #1487/#1490 made `fetch` a shadowable platform:project cap and deleted the tunnel, but kept the DEFAULT pipe inside the Project DO (`ProjectEgress.call` → `egressFetch`). This PR replaces that terminal with the stateless **`EgressPipe`** loopback. The Project DO still supervises every dispatch (live shadows resolve in its registry), but egress secrets are D1 rows scoped by the registry-injected `projectId`, so substitution + the real outbound fetch run in a plain isolate and **secret material never enters the DO**. - **The Project DO has no fetch surface at all** — no `fetch`, no `ingressFetch`, no `egressFetch`. ### Worker-loading unification - `itx/isolate.ts` is the ONE place the platform's trust posture (Law 4 ITERATE scoping, Law 5 egress outbound) is wired into loaded isolates; the registry's source caps and the project worker both use it. (The Workers-RPC-safe `onRpcBroken` guard this PR carried shipped independently in #1490 — main's version adopted.) ### `ProjectCapability` dissolved The hand-wired forwarder entrypoint is deleted; nothing called it. ### Auth is the ONLY project-id minter New auth internal route `POST /internal/project/mint-project-id` (service-authed); OS operator/recovery creates (project directory + `itx.projects.create`) round-trip through it. `mintProjectId` is deleted from OS — the `prj_` id space has exactly one source. ### Legacy afterAppend/runner-state deleted The agent, slack-agent, slack-integration, and repo DOs lose their `afterAppend` RPCs and fake runner shapes (delivery has been on the host model for a while). Agent runtime state is now the honest `{ agentPath, processors: { [slug]: snapshot } }`; slack `ensureReady` returns a plain snapshot; the agent-stream benchmark updated. ## Deferred to main's posture (from the original plan) - `project` stays a hardwired built-in (per #1482's kernel choice) rather than a durable-object default; `DIALABLE_DURABLE_OBJECTS` stays empty by default (config-gated). - The egress cap is named `fetch` (per #1487), not `egress`. ## ⚠️ Merge order **#1489 must merge (and auth deploy) first** — this PR's create paths round-trip id minting through auth's new `/internal/project/mint-project-id`, and previews point at production auth. The preview e2e here 404s until that endpoint is live. ## Breaking changes (intended) - Agent `runtimeState` shape changed (consumers were shape-agnostic or updated). - `egressFetch` is gone from every surface; use `itx.fetch` / the `egress` cap. ## Testing - Full repo gates green (typecheck, lint, 35/35 apps/os test files). - Workers suites: project-ingress 6/6 (incl. live-shadow + revoke-restores-default), itx-stream-subscribe 13/13. - `project-mcp-server-connection` fails 2/3 **identically on the branch base** (verified in a clean worktree) — pre-existing. - Preview e2e exercises: the egress capability over capnweb (explicit + implicit doors), the new live-shadow helper, and auth-routed minting. ## Out of scope - Egress policy-as-data / hold-for-approval (the §9 follow-on). - Stream processors taking a synchronous SQL client (jam). 🤖 Generated with [Claude Code](https://claude.com/claude-code)  --- > [!NOTE] > **High Risk** > Breaking egress and secret-handling semantics (DO no longer substitutes secrets; interceptors see raw placeholders), new auth dependency for id minting, and changed agent runtimeState shape affect security-sensitive paths and deploy ordering. > > **Overview** > Completes **itx D23**: project egress is a shadowable **`fetch`** capability whose default terminal is the stateless **`EgressPipe`** (secret substitution + outbound fetch in a plain isolate), while the Project DO only supervises registry dispatch. **`fetch` / `egressFetch` are removed** from the Project DO; **`ProjectCapability`** is deleted. > > Adds **`itx/isolate.ts`** so project workers, source caps, and the run harness share one **ITERATE + `ProjectEgress` globalOutbound** wiring path. > > **Auth is the sole `prj_` minter**: OS drops local **`mintProjectId`**; operator/admin and **`itx.projects.create`** call auth’s **`mintProjectId`** internal route. > > Removes legacy **`afterAppend`** / runner-shaped RPCs on agent, slack, and repo DOs; agent **`runtimeState`** is **`{ agentPath, processors }`** (benchmark updated). Docs mark §8/§9 shipped; live **`fetch`** shadows see **raw** `getSecret(...)` placeholders (withheld-text mode removed). > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit df5965b. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup>   ## Environment Config Lease    Lease: `preview-6` Doppler config: `preview_6` Type: `environment-config-lease` Leased until: 2026-06-11T11:19:46.363Z ### OS Status: deployed Commit: `df5965b` Preview: https://os.iterate-preview-6.com [Workflow run](https://github.com/iterate/iterate/actions/runs/27340067055) Updated: 2026-06-11T10:23:48.699Z  --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

jonastemplestein merged commit a52c5a1 into main Jun 11, 2026
8 checks passed

jonastemplestein deleted the itx-followups branch June 11, 2026 07:08

jonastemplestein mentioned this pull request Jun 11, 2026

itx grand cleanup: §8+§9 graduate — egress is a capability, captun intercept dies, kernel shrinks, auth mints, legacy afterAppend deleted #1485

Merged

mmkal mentioned this pull request Jun 11, 2026

Migrate captun to published 0.0.3 #1439

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

itx follow-ups: intercept tunnel deleted, streams is a cap, scoped DO names, REPL reads types.ts#1490

itx follow-ups: intercept tunnel deleted, streams is a cap, scoped DO names, REPL reads types.ts#1490
jonastemplestein merged 1 commit into
mainfrom
itx-followups

jonastemplestein commented Jun 11, 2026 •

edited by iterate-bot

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jonastemplestein commented Jun 11, 2026 • edited by iterate-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1. The egress intercept tunnel is deleted — shadowing was already the feature

2. streams joins the platform defaults

3. Durable-object dials are name-scoped

4. The REPL editor consumes types.ts — drift is now structurally impossible

The kernel after this PR

Verification

Environment Config Lease

OS

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jonastemplestein commented Jun 11, 2026 •

edited by iterate-bot

Loading

2. `streams` joins the platform defaults