Skip to content

Commit 6ace302

Browse files
wzhgbawzhgba
committed
feat(training-export): overhaul trigger system and message conversion
1 parent d16fff1 commit 6ace302

11 files changed

Lines changed: 1957 additions & 2 deletions

‎docs/training-export.md‎

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
---
2+
summary: "Opt-in runtime training data export that produces episode-level JSONL from compaction, session reset, and trajectory export triggers"
3+
read_when:
4+
- Enabling or configuring the training export feature
5+
- Understanding the trigger architecture and episode format
6+
- Debugging training export output or missing episodes
7+
- Reviewing the privacy and retention implications of unredacted training data
8+
title: "Training Export"
9+
---
10+
11+
# Training Export
12+
13+
## Overview
14+
15+
The training export system produces episode-level JSONL training data from the OpenClaw runtime trajectory. Each line in the output file is a self-contained training sample — either a **task episode** capturing an agent turn, or a **compact-summary episode** capturing how the agent compresses conversation context.
16+
17+
**Output:** `~/.openclaw/training-export/episodes.jsonl`
18+
19+
### Design Principles
20+
21+
- **Trajectory-first.** All training-required fields (system prompt, messages, tools, model metadata) are sourced from runtime trajectory events, not reconstructed offline.
22+
- **Trigger-driven.** Export is invoked at well-defined trigger points (compaction hooks, session reset, manual export command). No separate offline pipeline.
23+
- **Provider-owned conversion.** Message and tool conversion delegates to the Pi SDK / provider layer wherever possible, minimizing duplicated conversion logic.
24+
- **Unified compaction hook.** Training export for all compaction modes streams through a single pair of Pi SDK hooks (`session_before_compact` + `session_compact`), rather than being called from individual compaction paths.
25+
- **Pair-export guarantee.** For compaction-triggered exports, a task episode and a compact-summary episode must appear as a complete pair. If either is filtered by quality checks, the entire batch is discarded.
26+
- **Config-gated at call sites.** Callers check `trainingExport.enabled` before invoking export, so the intent is visible at every entry point without digging into implementation details.
27+
28+
---
29+
30+
## Trigger Architecture
31+
32+
### Compaction Hooks (Primary Trigger)
33+
34+
The training export extension registers two Pi SDK hooks that fire for **all** compaction modes:
35+
36+
| Hook | When | Action | Coverage |
37+
| ------------------------ | -------------------------- | --------------------------------------------------------------- | --------------------------------------------- |
38+
| `session_before_compact` | Before compaction executes | Stash pre-compaction snapshot + Pi SDK `preparation` (no write) | default, safeguard, manual |
39+
| `session_compact` | After compaction completes | Validate summary, build task + summary pair, write | default, safeguard, manual, overflow, timeout |
40+
41+
**Pair-export flow:**
42+
43+
1. `session_before_compact` — stash the current runtime snapshot and Pi SDK's `preparation` object (which provides `messagesToSummarize`, `previousSummary`, and `customInstructions`)
44+
2. `session_compact`:
45+
- If the compaction summary is empty (boundary-only compaction where `keepRecentTokens` covers all messages) → discard stash, no export
46+
- If the summary is valid → build both episodes; if **either** is filtered by quality checks, discard the entire batch
47+
- On success → atomically write both episodes
48+
49+
### Non-Compaction Triggers
50+
51+
| Trigger | Call Site | Exports |
52+
| ------------------- | ---------------------------------------------------- | ------------ |
53+
| `before_reset` | `src/gateway/session-reset-service.ts` | task episode |
54+
| `trajectory_export` | `src/auto-reply/reply/commands-export-trajectory.ts` | task episode |
55+
56+
Both call sites guard on `getTrainingExportConfig(cfg)?.enabled === true` before calling `runTrainingExport`.
57+
58+
### Extension Registration
59+
60+
The extension is registered in `src/agents/pi-embedded-runner/extensions.ts`, gated on `trainingExport.enabled`:
61+
62+
```typescript
63+
if (getTrainingExportConfig(params.cfg)?.enabled === true) {
64+
setCompactionTrainingExportRuntime(params.sessionManager, params.cfg ?? null);
65+
factories.push(compactionTrainingExportExtension);
66+
}
67+
```
68+
69+
---
70+
71+
## Episode Types
72+
73+
### Task Episode
74+
75+
Triggered by `on_compaction` (without `compactionEntry`), `before_reset`, or `trajectory_export`.
76+
77+
Built from the runtime snapshot collected from the latest `context.compiled` trajectory event:
78+
79+
- System prompt
80+
- Runtime messages (with trailing non-assistant messages trimmed for `on_compaction` — see below)
81+
- Runtime tools
82+
- Model metadata and trace info
83+
84+
**Trailing trim.** For all trigger types, if the snapshot ends mid-turn at a non-`assistant` message (e.g. `toolResult`), trailing non-`assistant` messages are removed. The `trainExampleMessagesAreUsable` check requires ≥1 user + ≥1 assistant; if trimming leaves the episode unusable, it is discarded. This is a training-data quality requirement, not a trigger-specific behavior.
85+
86+
### Compact-Summary Episode
87+
88+
Triggered by `on_compaction` (with `compactionEntry`).
89+
90+
Payload is built from the Pi SDK `preparation` object stashed during `session_before_compact`:
91+
92+
| Field | Source |
93+
| -------------- | ----------------------------------------------------------------------------------------- |
94+
| `systemPrompt` | `COMPACT_SUMMARIZATION_SYSTEM_PROMPT` (local constant) |
95+
| `promptText` | `buildCompactSummaryPrompt({ messagesToSummarize, previousSummary, customInstructions })` |
96+
| `responseText` | `compactionEntry.summary` |
97+
| `compaction` | `tokensBefore`, `firstKeptEntryId`, `fromExtension` |
98+
99+
**Empty-summary guard.** When `messagesToSummarize` is empty (short conversations where `keepRecentTokens` covers all messages), `serializeCompactSummaryConversation` returns an empty string, producing `<conversation>\n\n</conversation>`. The `compactConversationTextIsNonEmpty` regex (`/<conversation>\s*[\s\S]*\S[\s\S]*<\/conversation>/`) requires at least one non-whitespace character between the tags, so the summary episode fails validation and is filtered. Combined with pair-export, both task and summary episodes are correctly discarded for this boundary case.
100+
101+
---
102+
103+
## Message Conversion Pipeline
104+
105+
Messages are converted via the `chat_completions` format pipeline:
106+
107+
```
108+
runtime messages (Pi SDK format)
109+
↓
110+
1. Pre-process (single map over messages)
111+
a. Strip thinking blocks from assistant messages
112+
b. Convert compactionSummary → user message
113+
↓
114+
2. Upstream convertMessages() from @mariozechner/pi-ai/openai-completions
115+
↓
116+
3. adaptChatCompletionsMessagesToExportMessages()
117+
↓
118+
4. Append reasoning_content (scanned from original runtimeMessages)
119+
↓
120+
5. developer role → system role (training format compatibility)
121+
```
122+
123+
### Why CompactionSummary Conversion is Needed
124+
125+
Pi SDK's `convertToLlm` (`@mariozechner/pi-coding-agent/dist/core/messages.js:103-108`) converts `compactionSummary` messages to user messages with wrapper text. However, the upstream `convertMessages` from `@mariozechner/pi-ai/openai-completions` does **not** handle the `compactionSummary` role. Without pre-processing, compaction summary messages are silently dropped from task episodes, losing critical context.
126+
127+
The pre-processing step mirrors Pi SDK's conversion format:
128+
129+
```
130+
The conversation history before this point was compacted into the following summary:
131+
132+
<summary>
133+
{summary text}
134+
</summary>
135+
```
136+
137+
---
138+
139+
## Configuration
140+
141+
```typescript
142+
interface TrainingExportConfig {
143+
enabled?: boolean; // default: false (opt-in)
144+
compat?: ModelCompatConfig; // model compatibility overrides for export
145+
}
146+
```
147+
148+
The `enabled` check is applied at every entry point (call sites + extension registration).
149+
150+
---
151+
152+
## Trigger Types
153+
154+
| Kind | Scenario | Distinction |
155+
| ------------------- | --------------------- | ----------------------------------------------------------------- |
156+
| `on_compaction` | Compaction event | Has `compactionEntry` → summary episode; otherwise → task episode |
157+
| `before_reset` | Session reset | task episode |
158+
| `trajectory_export` | Manual export command | task episode |
159+
160+
---
161+
162+
## Key Files
163+
164+
| File | Responsibility |
165+
| ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
166+
| `src/training-export.ts` | Core: snapshot collection, episode construction, JSONL I/O, prompt constants, compaction extension (merged from `compaction-summary-prompt.ts` and `compaction-training-export.ts`) |
167+
| `src/agents/pi-embedded-runner/extensions.ts` | Extension registration (config-gated) |
168+
| `src/agents/pi-hooks/compaction-safeguard.ts` | Safeguard compaction logic (no longer contains training export calls) |
169+
| `src/agents/pi-embedded-runner/compact.ts` | Manual compaction (no longer contains training export calls) |
170+
| `src/gateway/session-reset-service.ts` | `before_reset` trigger → `runTrainingExport` |
171+
| `src/auto-reply/reply/commands-export-trajectory.ts` | `trajectory_export` trigger → `runTrainingExport` |
172+
173+
---

‎src/agents/openai-transport-stream.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -214,7 +214,7 @@ function parseTextSignature(
214214
return { id: signature };
215215
}
216216

217-
function convertResponsesMessages(
217+
export function convertResponsesMessages(
218218
model: Model<Api>,
219219
context: Context,
220220
allowedToolCallProviders: Set<string>,
@@ -386,7 +386,7 @@ function convertResponsesMessages(
386386
return messages;
387387
}
388388

389-
function convertResponsesTools(
389+
export function convertResponsesTools(
390390
tools: NonNullable<Context["tools"]>,
391391
model: OpenAIModeModel,
392392
options?: { strict?: boolean | null },

‎src/agents/pi-embedded-runner/extensions.ts‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,11 @@ import type { AgentToolResult } from "@mariozechner/pi-agent-core";
33
import type { ExtensionFactory, SessionManager } from "@mariozechner/pi-coding-agent";
44
import type { OpenClawConfig } from "../../config/types.openclaw.js";
55
import type { ProviderRuntimeModel } from "../../plugins/provider-runtime-model.types.js";
6+
import {
7+
compactionTrainingExportExtension,
8+
getTrainingExportConfig,
9+
setCompactionTrainingExportRuntime,
10+
} from "../../training-export.js";
611
import { resolveContextWindowInfo } from "../context-window-guard.js";
712
import { DEFAULT_CONTEXT_TOKENS } from "../defaults.js";
813
import { createAgentToolResultMiddlewareRunner } from "../harness/tool-result-middleware.js";
@@ -156,6 +161,11 @@ export function buildEmbeddedExtensionFactories(params: {
156161
});
157162
factories.push(compactionSafeguardExtension);
158163
}
164+
// Register compaction training export only when enabled.
165+
if (getTrainingExportConfig(params.cfg)?.enabled === true) {
166+
setCompactionTrainingExportRuntime(params.sessionManager, params.cfg ?? null);
167+
factories.push(compactionTrainingExportExtension);
168+
}
159169
const pruningFactory = buildContextPruningFactory(params);
160170
if (pruningFactory) {
161171
factories.push(pruningFactory);

‎src/auto-reply/reply/commands-export-trajectory.ts‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import type { ExecToolDetails } from "../../agents/bash-tools.js";
44
import { formatErrorMessage } from "../../infra/errors.js";
55
import type { ExecApprovalRequest } from "../../infra/exec-approvals.js";
66
import { pathExists } from "../../infra/fs-safe.js";
7+
import { getTrainingExportConfig, runTrainingExport } from "../../training-export.js";
78
import {
89
exportTrajectoryForCommand,
910
formatTrajectoryCommandExportSummary,
@@ -169,6 +170,18 @@ export async function buildExportTrajectoryReply(
169170
};
170171
}
171172

173+
if (getTrainingExportConfig(params.cfg)?.enabled === true) {
174+
runTrainingExport({
175+
trigger: {
176+
kind: "trajectory_export",
177+
sessionId: entry.sessionId,
178+
sessionFile,
179+
command: params.command.commandBodyNormalized,
180+
},
181+
config: params.cfg,
182+
});
183+
}
184+
172185
return {
173186
text: formatTrajectoryCommandExportSummary(summary),
174187
};

‎src/config/schema.help.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -643,6 +643,10 @@ export const FIELD_HELP: Record<string, string> = {
643643
"Include full message payloads in trace output (default: true).",
644644
"diagnostics.cacheTrace.includePrompt": "Include prompt text in trace output (default: true).",
645645
"diagnostics.cacheTrace.includeSystem": "Include system prompt in trace output (default: true).",
646+
"trainingExport.enabled":
647+
"Enable or disable built-in training export at trigger points driven by session semantics and explicit trajectory export commands (compaction, reset, and trajectory export).",
648+
"trainingExport.compat":
649+
"Optional model-compat override applied only to training export when calling provider-owned converters from collected trajectory snapshots. Reuses the system ModelCompatConfig shape.",
646650
"tools.exec.applyPatch.enabled":
647651
"Enable or disable apply_patch for OpenAI and OpenAI Codex models when allowed by tool policy (default: true).",
648652
"tools.exec.applyPatch.workspaceOnly":

‎src/config/schema.labels.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,8 @@ export const FIELD_LABELS: Record<string, string> = {
6767
"diagnostics.cacheTrace.includeMessages": "Cache Trace Include Messages",
6868
"diagnostics.cacheTrace.includePrompt": "Cache Trace Include Prompt",
6969
"diagnostics.cacheTrace.includeSystem": "Cache Trace Include System",
70+
"trainingExport.enabled": "Training Export Enabled",
71+
"trainingExport.compat": "Training Export Compat Override",
7072
"agents.list.*.identity.avatar": "Identity Avatar",
7173
"agents.list.*.skills": "Agent Skill Filter",
7274
"agents.list[].runtime": "Agent Runtime",

‎src/config/types.openclaw.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ import type {
2424
CommandsConfig,
2525
MessagesConfig,
2626
} from "./types.messages.js";
27+
import type { ModelCompatConfig } from "./types.models.js";
2728
import type { ModelsConfig } from "./types.models.js";
2829
import type { NodeHostConfig } from "./types.node-host.js";
2930
import type { PluginsConfig } from "./types.plugins.js";
@@ -129,6 +130,10 @@ export type OpenClawConfig = {
129130
cron?: CronConfig;
130131
commitments?: CommitmentsConfig;
131132
hooks?: HooksConfig;
133+
trainingExport?: {
134+
enabled?: boolean;
135+
compat?: ModelCompatConfig;
136+
};
132137
discovery?: DiscoveryConfig;
133138
talk?: TalkConfig;
134139
gateway?: GatewayConfig;

‎src/config/zod-schema.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -441,6 +441,13 @@ export const OpenClawSchema = z
441441
})
442442
.strict()
443443
.optional(),
444+
trainingExport: z
445+
.object({
446+
enabled: z.boolean().optional(),
447+
compat: z.object({}).passthrough().optional(),
448+
})
449+
.strict()
450+
.optional(),
444451
logging: z
445452
.object({
446453
level: LoggingLevelSchema.optional(),

‎src/gateway/session-reset-service.ts‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ import {
3737
normalizeAgentId,
3838
parseAgentSessionKey,
3939
} from "../routing/session-key.js";
40+
import { getTrainingExportConfig, runTrainingExport } from "../training-export.js";
4041
import { ErrorCodes, errorShape } from "./protocol/index.js";
4142
import {
4243
archiveSessionTranscriptsDetailed,
@@ -451,6 +452,7 @@ export async function emitGatewayBeforeResetPluginHook(params: {
451452
const sessionFile = params.entry?.sessionFile;
452453
const agentId = normalizeAgentId(params.target.agentId ?? resolveDefaultAgentId(params.cfg));
453454
const workspaceDir = resolveAgentWorkspaceDir(params.cfg, agentId);
455+
454456
let messages: unknown[] = [];
455457
try {
456458
if (typeof sessionId === "string" && sessionId.trim().length > 0) {
@@ -636,6 +638,19 @@ export async function performGatewaySessionReset(params: {
636638
store[primaryKey] = nextEntry;
637639
return nextEntry;
638640
});
641+
// Training export for before_reset trigger — independent of plugin hooks.
642+
if (getTrainingExportConfig(cfg)?.enabled === true) {
643+
runTrainingExport({
644+
trigger: {
645+
kind: "before_reset",
646+
sessionId: resetSourceEntry?.sessionId,
647+
sessionFile: resetSourceEntry?.sessionFile,
648+
reason: params.reason,
649+
},
650+
config: cfg,
651+
});
652+
}
653+
639654
await emitGatewayBeforeResetPluginHook({
640655
cfg,
641656
key: params.key,

0 commit comments

Comments
 (0)