agents: per-project agent context — the project config worker is a stream processor#1475
Merged
Conversation
The project's config worker is now a stream processor: a new project-config-worker processor (hosted on ProjectDurableObject, subscribed to the project root stream) forwards every committed event to the config worker's afterAppend export. Project code reacts to facts by appending facts — no bespoke hook protocol, no merge logic. The motivating use case is per-project agent context, the original goal of the agents-system work: the root stream carries child-stream-created for every new stream in the project, so a config worker can watch for new /agents/... paths and append its own system-prompt-updated, capability-noted, or llm-config-updated events to them. Precedence and refresh need no machinery: last-wins reducers and the existing defaults-yield-to-existing-events behavior in ensureAgentSetupEvents resolve both orderings of the creation race. Mechanics: - The forwarding processor delivers under blockProcessorWhile: ordered, checkpointed, at-least-once. The DO-side forward swallows user-code failures (a throwing afterAppend must never wedge root-stream delivery) and no-ops until the config worker's first build (provisioning does that within seconds of project creation). - Entrypoint resolution for forwarding prefers correctness over latency: unlike the ingress path (which serves the stale cached worker while rebuilding), a stale checkout AWAITS the rebuild so a just-pushed config sees the very next event instead of losing the one that triggered it. - The subscription is ensured on every project wake, so projects created before this processor existed get subscribed too. - afterAppend on object-syntax worker exports receives (input, env) — the workerd-level test proves it, and env.STREAMS.append is the write path. Tests: unit (forwarding order, failed forwards do not advance the checkpoint), workerd (full chain: subscription wiring, blocking forward, fresh-entrypoint resolution, env argument — the config worker echoes a root-stream ping back as a fact), and a deterministic e2e (an injected agent script pushes an afterAppend config worker; a fresh agent path then wakes and its stream must show the custom prompt as the LAST system-prompt fact plus the announced capability). The iterate-config template documents the pattern with a commented example. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
45e3c4a to
f41a89a
Compare
afterAppend was the dead runner-model name (the vestigial surface this PR gave a real driver happened to carry it, which propagated legacy vocabulary into a brand-new API). The worker-facing hook now matches the StreamProcessor class model: processEvent, with the stream path passed alongside the event instead of baked into a legacy event shape. The project DO's uncalled public afterAppend method is deleted outright (it existed only for a debug surface that no longer references it). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Addresses the Cursor Bugbot finding on #1475 (stale config within refresh window): the forwarder trusted the ingress path's 10s freshness window, so a config push landing right after a rebuild served the previous worker — and since an event can be the direct consequence of a push (a new agent created right after its config landed), the old worker would consume the very trigger the new config exists to handle. Lost, not delayed. readRemoteBranchOid resolves the branch head with ONE smart-HTTP request (the git ls-remote ref advertisement; pkt-line parse, no clone) and the forwarder compares it to the cached checkout's commit: match → cached entrypoint, mismatch → await the rebuild. Probe failure falls back to the time-based window rather than blocking delivery on repo availability. Rebuilds now only happen when the repo actually changed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 89c468b. Configure here.
…are swallowed Addresses the Cursor Bugbot finding on #1475 (platform forward errors skip retry): one try/catch wrapped both entrypoint resolution (including awaited rebuilds) and the user's processEvent, swallowing everything — so a transient repo/build failure completed the forward 'successfully', the checkpoint advanced, and the event was silently dropped despite the blocking at-least-once design. Split by fault: platform failures (resolution/rebuild) now propagate, so the blocking delivery holds the checkpoint and the event is redelivered; only the project author's processEvent throwing is swallowed and logged, since user bugs must never wedge root-stream delivery into the poison-batch disconnect. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jonastemplestein
added a commit
that referenced
this pull request
Jun 10, 2026
Convergent with this branch's forwarding, but main's version is the real
contract — checkpointed at-least-once delivery via a dedicated
project-config-worker processor — so it wins. Ported into the refactored
structure:
- The DO hosts main's ProjectConfigWorkerProcessor as a second processor;
ensureProjectSubscription configures both subscriptions.
- forwardEventToWorker carries main's failure split (user hook errors
swallowed, platform errors hold the checkpoint) and exact-freshness
resolution: ls-remote HEAD vs cached commit, awaited rebuild on mismatch,
time-window fallback on probe failure — implemented on WorkerHost
(new currentBuild()/checkoutIsFresh() accessors).
- The worker hook is processEvent({ event, streamPath }); this branch's
best-effort forwarding inside ProjectProcessor is deleted (and its "*"
consumes with it).
- Base seed keeps main's stream-processor framing with this branch's
"worker" vocabulary; main's end-to-end forwarding test passes against the
ported implementation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jonastemplestein
added a commit
that referenced
this pull request
Jun 10, 2026
…1483) Steps B and C of the agents roadmap (after #1460 and #1475): the LLM request handoff stops embedding the conversation, and the two LLM request processors become deliberate, tidy siblings sharing pure helpers. ## Request-by-reference (no more embedded body) `agent/llm-request-requested` used to carry the full chat request. Since the conversation grows with the stream, every request stored a complete copy of it — O(N²) stream growth. The `llmRequestId` already IS the requested event's offset, so the body is redundant: providers can rebuild it from committed history. Before: ```ts // agent processor, on handoff payload: { model, runOpts, body: buildLlmChatRequest(stateAtRequest), // full conversation, every time } ``` After: ```ts // agent processor: just the reference + how to run it payload: { model: stateAtRequest.llmConfig.model, runOpts: stateAtRequest.llmConfig.runOpts } // provider, at execution time (both cloudflare-ai and openai-ws): // Request-by-reference: the requested event carries no body; rebuild the // chat request from committed history up to the request's own offset. const body = buildAgentLlmRequestBody({ events: await this.deps.readStreamEvents(), llmRequestId, // === the requested event's offset }); ``` The rebuild reduces history `events.filter((e) => e.offset <= llmRequestId)` through the same `reduceAgentEvents` + `buildLlmChatRequest` pair the agent itself uses, so the model-visible context is reproducible from the stream forever — including for crash-recovery retries, which re-derive exactly what the dead incarnation would have sent. **Breaking change** to the `agent/llm-request-requested` payload (and `cloudflare-ai/llm-request-started`, which also embedded the body). No backcompat bridge; prd gets redeployed. ## Providers as siblings, not an abstraction `cloudflare-ai` and `openai-ws` were ~500/790-line copy-pasted state machines. Rather than an abstract base class, they're now deliberate siblings: same method names, same control flow, same comments where the logic matches — each keeps its own event types (which scales better as providers diverge) and its own transport (one `AI.run()` call vs a shared Responses WebSocket). What they share are four stateless functions in `llm-request-helpers.ts`: ```ts buildAgentLlmRequestBody({ events, llmRequestId }); // the request-by-reference rebuild isAgentLlmRequestStillCurrent({ events, llmRequestId }); // stale-output guard before agent-visible appends findDanglingLlmRequestIds({ requests, executedLlmRequestIds }); // crash-recovery candidates parseLlmRequestRequestedEventAt({ events, llmRequestId }); // typed re-derivation for recovery ``` Both implementation files open with the same note: *"When you fix something here, check whether the sibling needs the same fix."* ## Bounded execution-claims set `#executedLlmRequestIds` (the instance-scoped set distinguishing "this incarnation is executing it" from "a dead one was") previously only grew. Both siblings now drop a claim when the request's own completed fact reduces back: ```ts case "events.iterate.com/cloudflare-ai/llm-request-completed": // The completed fact is durable; this instance can never need to // (re-)execute this request again, so drop the claim — this is what // keeps the executed set bounded. this.#executedLlmRequestIds.delete(event.payload.llmRequestId); return; ``` ## Tests - New in both provider suites: *rebuilds the chat request from history up to the request's offset* — history rows after the requested event's offset are excluded from what the model sees. - The agent handoff test now asserts the requested payload is `{ model, runOpts }` with no `body`. - Provider fixtures lost their embedded bodies; conversation content now flows through `readStreamEvents` history, matching production. `pnpm typecheck && pnpm lint && pnpm format && pnpm test` all green. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Breaking event payload shapes for `llm-request-requested` and provider started events; correctness now depends on history reads and offset-bounded rebuild at execution time, including crash recovery paths. > > **Overview** > **Request-by-reference LLM handoff** stops embedding the full conversation on `agent/llm-request-requested` (and drops `body` from `cloudflare-ai/llm-request-started`). Handoffs are now `{ model, runOpts }` only; `llmRequestId` stays the requested event’s offset, and **cloudflare-ai** / **openai-ws** rebuild model input at execution via shared `llm-request-helpers.ts` (`buildAgentLlmRequestBody` reduces history with `offset <= llmRequestId`). > > The two provider processors are aligned as **siblings** (shared helpers for rebuild, still-current checks, dangling recovery, typed re-parse) instead of duplicated logic. **`#executedLlmRequestIds`** now drops entries when each request’s provider **completed** event reduces, so long-lived instances don’t grow the claim set forever. > > Tests assert handoff payloads have no `body` and that providers exclude stream rows after the request offset when building chat input. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 83ed554. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- CLOUDFLARE_PREVIEW --> ## Environment Config Lease <!-- CLOUDFLARE_PREVIEW_STATE --> <!-- { "apps": { "os": { "appDisplayName": "OS", "appSlug": "os", "status": "deployed", "updatedAt": "2026-06-10T22:06:49.277Z", "headSha": "83ed55497ec2f4d04aeb172f36d2dddd34b8dcfc", "message": null, "publicUrl": "https://os.iterate-preview-7.com", "runUrl": "https://github.com/iterate/iterate/actions/runs/27309177659", "shortSha": "83ed554" } }, "environmentConfigLease": { "dopplerConfig": "preview_7", "leasedUntil": 1781132576419, "leaseId": "7f1ba814-5c43-4cdd-ae72-2f569050a9d5", "slug": "preview-7", "type": "environment-config-lease" } } --> <!-- /CLOUDFLARE_PREVIEW_STATE --> Lease: `preview-7` Doppler config: `preview_7` Type: `environment-config-lease` Leased until: 2026-06-10T23:02:56.419Z ### OS Status: deployed Commit: `83ed554` Preview: https://os.iterate-preview-7.com [Workflow run](https://github.com/iterate/iterate/actions/runs/27309177659) Updated: 2026-06-10T22:06:49.277Z <!-- /CLOUDFLARE_PREVIEW --> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
jonastemplestein
added a commit
that referenced
this pull request
Jun 10, 2026
The last batch of outstanding work from the agents audit (after #1460, #1475, #1483): the section-3 dead-code sweep and the UI shrink. Net **−188 lines** (−503/+315). ## One agent setup form `agents/new.tsx` and `agents/new-preset.tsx` were ~270-line, ~90%-identical forms (provider/model/runOpts/system-prompt/custom-events fields plus the YAML preview pane). They now share one `AgentSetupFormPage` component; each route keeps only what genuinely differs — its path normalization, its preview builder, and its submit mutation: ```tsx <AgentSetupFormPage title="New Agent" pathLabel="Agent path" buildPreview={(values) => buildPreviewEvents({ projectId: project.id, values })} submitIdleLabel="Create agent" isPending={createAgent.isPending} onSubmit={({ preview }) => createAgent.mutate(preview)} ... /> ``` The routes drop from ~270 lines each to ~125, and the next form tweak happens once instead of twice. ## Legacy Slack preset filter deleted `agent-presets.ts` carried `isLegacyGeneratedSlackOpenAiPreset` — a content-sniffing filter that suppressed an old auto-generated `/agents/slack` preset by matching its system-prompt text. Checked prd before deleting: the iterate project has **zero** stored presets, so the filter guards nothing. The intentional behavior next to it (Slack agents never inherit the generic `/agents` preset) stays, with its tests. ## Stale migration headers Every processor under `apps/os/src/domains/{agents,slack}/stream-processors/` opened with "Migrated from `packages/shared/src/stream-processors/...`" — a directory that no longer exists. Those provenance paragraphs are gone. Where they carried a live constraint, the constraint survives in its own words: ```ts // Appended event types, payload shapes, and idempotency-key derivations // (`agent/<key>@<sourceOffset>`) are stable wire formats — changing them // breaks dedup against events already committed to streams. ``` ## Audit bug status (no code change needed) - **2.4 zombie `pendingTriggerCount`** — fixed by construction since #1460: the reconcilers guarantee dangling requests reach a terminal event, and a queued count only ever represents real user inputs whose follow-up turn rebuilds from full history. - **2.3 cancellation check-then-act race** — still a theoretical window; closing it needs conditional appends (append-if-still-current at the stream layer), which is its own design, not a cleanup. `pnpm typecheck && pnpm lint && pnpm format && pnpm test` all green. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Mostly UI deduplication and comment edits; Slack preset selection is slightly broader if old auto-generated presets exist in storage, which the PR assumes is empty. > > **Overview** > Introduces **`AgentSetupFormPage`** so **New Agent** and **New Agent Preset** share one form (provider, model, run options, system prompt, custom events YAML, live preview). Each route only keeps path handling, its preview builder, and submit logic—roughly halving page size. > > **Removes** the legacy **`isLegacyGeneratedSlackOpenAiPreset`** filter and its test from `agent-presets.ts`. Slack agents still only match Slack-scoped presets; stored `/agents/slack` presets are no longer sniffed and ignored by prompt text. > > **Trims** stale “migrated from `packages/shared`…” headers across agent and Slack stream-processor modules, leaving short notes where wire formats and idempotency keys must stay stable. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit c9cf738. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- CLOUDFLARE_PREVIEW --> ## Environment Config Lease <!-- CLOUDFLARE_PREVIEW_STATE --> <!-- { "apps": { "os": { "appDisplayName": "OS", "appSlug": "os", "status": "deployed", "updatedAt": "2026-06-10T22:24:45.981Z", "headSha": "c9cf738e2dad4877de8286cf370cf50cedd81eeb", "message": null, "publicUrl": "https://os.iterate-preview-4.com", "runUrl": "https://github.com/iterate/iterate/actions/runs/27310094398", "shortSha": "c9cf738" } }, "environmentConfigLease": { "dopplerConfig": "preview_4", "leasedUntil": 1781133683378, "leaseId": "51ccdfaf-05f8-4f1c-a8ed-1b26dec7500b", "slug": "preview-4", "type": "environment-config-lease" } } --> <!-- /CLOUDFLARE_PREVIEW_STATE --> Lease: `preview-4` Doppler config: `preview_4` Type: `environment-config-lease` Leased until: 2026-06-10T23:21:23.378Z ### OS Status: deployed Commit: `c9cf738` Preview: https://os.iterate-preview-4.com [Workflow run](https://github.com/iterate/iterate/actions/runs/27310094398) Updated: 2026-06-10T22:24:45.981Z <!-- /CLOUDFLARE_PREVIEW --> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
jonastemplestein
added a commit
that referenced
this pull request
Jun 11, 2026
Post-merge grooming after the agents workstream landed (#1460, #1475, #1483, #1484). Grooming rules (docs/tasks-grooming.md) say tasks are deleted when done: - **Deleted** `tasks/streams-core-processor-host-homogenization.md` — the plan of record for what shipped in #1460. - **Deleted** `tasks/agents-system-audit-and-reconciler-design.md` — the audit knowledge dump; every verified bug and design direction in it is now either shipped or carried by a live task file. - **Updated** the two deferred follow-ups (`streams-core-clock-durable-timers.md`, `streams-event-kinds-metadata.md`) to drop their `dependsOn`/background references to the deleted docs, pointing at the merged PRs instead. - **Added** `tasks/streams-conditional-appends.md` — the one audit finding that survived everything: the check-then-act window between a provider's still-current check and its `agent/output-added` append. Backlog, with the conditional-append direction written down so it isn't lost with the audit doc. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Documentation-only changes under `tasks/` with no runtime or API impact. > > **Overview** > **Grooms the `tasks/` backlog** after agents/streams work landed in #1460 and related PRs, per `docs/tasks-grooming.md` (delete tasks when done). > > **Removes** the shipped plan-of-record (`streams-core-processor-host-homogenization.md`) and the umbrella audit/knowledge dump (`agents-system-audit-and-reconciler-design.md`), since their content is either merged or split elsewhere. > > **Refreshes** deferred follow-ups: `streams-core-clock-durable-timers.md` and `streams-event-kinds-metadata.md` drop `dependsOn` on deleted tasks and cite PR #1460 in background instead of dead links. > > **Adds** `streams-conditional-appends.md` (backlog) to capture the remaining audit item—the LLM output **check-then-act** race—and the direction (stream-level conditional append / CAS), so it isn’t lost with the audit doc. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit d9b9d7f. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

What
A project's
iterate-config/worker.jsis now a stream processor. Every event committed to the project root stream (/) is delivered to its exportedprocessEventhook — same name and spirit as theStreamProcessorclass model — in order, checkpointed, at-least-once. Project code reacts to facts by appending facts. That one mechanism is the whole API: no hook protocol, no config schema, no merge logic.The motivating use case (and the original goal of the agents-system work,
tasks/agents-system-audit-and-reconciler-design.md§4): per-project agent context. Every new stream in a project — including every new agent — announces itself on the root stream asstream/child-stream-created. A config worker can watch for new agent paths and write its own context into them:The same shape works for model/provider config (
agent/llm-config-updated,os-agent/llm-provider-selected), primer rows (agent/input-addedwithdont-trigger-request), or reacting to anything else on the root stream (webhooks, lifecycle facts).How
A
processEvent-ish surface existed on the template (afterAppend) — but nothing in production drove it (only a debug route). This PR adds the missing processor:hosted on
ProjectDurableObjectnext toproject-lifecycle, subscribed to/withconsumes: ["*"](subscription ensured on every project wake, so pre-existing projects pick it up).Two deliberate policies on the DO side:
processEventis swallowed and logged — it must never poison root-stream delivery into a disconnect. Forwarding no-ops until the config worker's first build (project provisioning does that within seconds of creation).Testing
Unit (
project-config-worker/implementation.test.ts): events forwarded in stream order; a failed forward does not advance the checkpoint (the at-least-once replay delivers it).Workerd (
pnpm test:project-ingress): the full chain in real runtime — the test project's config worker echoes a root-stream ping back as a fact on another stream, proving subscription wiring, blocking forward, fresh-entrypoint resolution, and that object-syntax exports receive(input, env):E2E (deterministic, no LLM in the loop): an injected agent script pushes a
processEventconfig worker into the project's iterate-config repo; a fresh agent path then wakes, and its stream must show the custom prompt as the last system-prompt fact (i.e. what the agent actually runs with) plus the announced capability. Passed against the deployed preview.The iterate-config template (repo copy + base seed) documents the pattern with a live
processEventlogger plus a commented agent-customization example.pnpm typecheck && pnpm lint && pnpm format && pnpm testgreen.🤖 Generated with Claude Code
Environment Config Lease
No active environment config lease.
OS
Status: released
Commit:
6ae7b5ePreview: https://os.iterate-preview-6.com
Summary: Preview app released.
Workflow run
Updated: 2026-06-10T21:18:02.628Z
Note
Medium Risk
Changes how every project root-stream event is delivered to user code and how config worker builds are awaited on that path; mistakes could drop or delay customization events, though platform/user error split and tests mitigate this.
Overview
Project
iterate-config/worker.jsis wired as a stream processor: every event on the project root stream (/) is delivered toprocessEvent({ event, streamPath }, env), replacing the unusedafterAppendhook and removing the Project DO’s publicafterAppend.A new
project-config-workerprocessor onProjectDurableObjectsubscribes to/(idempotent subscription on each project wake) and forwards events in order underblockProcessorWhile, so checkpoints advance only after delivery. Platform failures (rebuild/resolution) throw for redelivery; userprocessEventthrows are logged and swallowed so root-stream processing cannot wedge.Event forwarding uses stricter config freshness than ingress: a single
readRemoteBranchOid(git ls-remote) compares remote HEAD to the cached checkout; on mismatch the forwarder awaits a rebuild instead of serving a stale worker while a push-triggered event is in flight.Templates/seeds document
processEventand a commented per-agent context pattern (child-stream-created→ append system prompt / capability on agent streams). Coverage adds unit forwarding/checkpoint tests, a workerd root-stream echo chain, and an e2e that pushes a custom config worker and asserts fresh agents get the last custom prompt and capability.Reviewed by Cursor Bugbot for commit 6ae7b5e. Bugbot is set up for automated code reviews on this repo. Configure here.