Skip to content

Commit 6e7254b

Browse files
westey-mwestey-m
authored
.NET: [BREAKING] Rename from ServiceStoredSimulatingChatClient to PerServiceCallChatHistoryPersistingChatClient (#4993)
* Rename from ServiceStoredSimulatingChatClient to PerServiceCallChatHistoryPersistingChatClient * Address PR comment
1 parent 9c57680 commit 6e7254b

13 files changed

Lines changed: 198 additions & 151 deletions

‎docs/decisions/0022-chat-history-persistence-consistency.md‎

Lines changed: 27 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ The persistence timing and `FunctionResultContent` trimming behaviors are interr
4242
## Considered Options
4343

4444
- Option 1: Per-run persistence with opt-in FRC (FunctionResultContent) trimming
45-
- Option 2: Opt-in per-service-call persistence (via `SimulateServiceStoredChatHistory`)
45+
- Option 2: Opt-in per-service-call persistence (via `RequirePerServiceCallChatHistoryPersistence`)
4646

4747
## Pros and Cons of the Options
4848

@@ -57,12 +57,12 @@ Keep the current default behavior of persisting chat history only at the end of
5757
- Bad, because if the process crashes mid-loop, all intermediate progress from the current run is lost, not satisfying driver C.
5858
- Bad, because this option alone does not provide a way for users to opt into per-service-call persistence, not satisfying driver E.
5959

60-
### Option 2: Opt-in per-service-call persistence (via `SimulateServiceStoredChatHistory`)
60+
### Option 2: Opt-in per-service-call persistence (via `RequirePerServiceCallChatHistoryPersistence`)
6161

62-
Introduce an optional SimulateServiceStoredChatHistory setting to persist chat history after each individual service call within the FIC loop, matching the AI service's behavior. Trailing `FunctionResultContent` trimming is unnecessary with this approach (it is naturally handled).
62+
Introduce an optional RequirePerServiceCallChatHistoryPersistence setting to persist chat history after each individual service call within the FIC loop, matching the AI service's behavior. Trailing `FunctionResultContent` trimming is unnecessary with this approach (it is naturally handled).
6363

6464
Settings:
65-
- `SimulateServiceStoredChatHistory` = `true`
65+
- `RequirePerServiceCallChatHistoryPersistence` = `true`
6666

6767
- Good, because the stored history matches the service's behavior when opting in for both timing and content, fully satisfying driver A.
6868
- Good, because intermediate progress is preserved if the process is interrupted, satisfying driver C.
@@ -73,36 +73,49 @@ Settings:
7373

7474
## Decision Outcome
7575

76-
Chosen option: **Option 2: Opt-in per-service-call persistence (via `SimulateServiceStoredChatHistory`)**. The existing per-run persistence behavior is retained as-is, requiring no changes from users. Per-service-call persistence is available as an opt-in feature via the `SimulateServiceStoredChatHistory` setting. This satisfies drivers B (atomicity) and D (simplicity) for the common case, while fully satisfying driver A (consistency) for users who opt into simulated service-stored behavior. Users who need per-service-call persistence for recoverability (driver C) can enable it explicitly.
76+
Chosen option: **Option 2: Opt-in per-service-call persistence (via `RequirePerServiceCallChatHistoryPersistence`)**. The existing per-run persistence behavior is retained as-is, requiring no changes from users. Per-service-call persistence is available as an opt-in feature via the `RequirePerServiceCallChatHistoryPersistence` setting. This satisfies drivers B (atomicity) and D (simplicity) for the common case, while fully satisfying driver A (consistency) for users who opt into simulated service-stored behavior. Users who need per-service-call persistence for recoverability (driver C) can enable it explicitly.
7777

7878
### Configuration Matrix
7979

80-
The behavior depends on the combination of `UseProvidedChatClientAsIs` and `SimulateServiceStoredChatHistory`:
80+
The behavior depends on the combination of `UseProvidedChatClientAsIs` and `RequirePerServiceCallChatHistoryPersistence`:
8181

82-
| `UseProvidedChatClientAsIs` | `SimulateServiceStoredChatHistory` | Behavior |
82+
| `UseProvidedChatClientAsIs` | `RequirePerServiceCallChatHistoryPersistence` | Behavior |
8383
|---|---|---|
8484
| `false` (default) | `false` (default) | **Per-run persistence.** Messages are persisted at the end of the full agent run via the `ChatHistoryProvider`. |
85-
| `false` | `true` | **Per-service-call persistence (simulated).** A `ServiceStoredSimulatingChatClient` middleware is automatically injected into the chat client pipeline between `FunctionInvokingChatClient` and the leaf `IChatClient`. Messages are persisted after each service call. A sentinel `ConversationId` causes FIC to treat the conversation as service-managed. |
85+
| `false` | `true` | **Per-service-call persistence (simulated).** A `PerServiceCallChatHistoryPersistingChatClient` middleware is automatically injected into the chat client pipeline between `FunctionInvokingChatClient` and the leaf `IChatClient`. Messages are persisted after each service call. A sentinel `ConversationId` causes FIC to treat the conversation as service-managed. |
8686
| `true` | `false` | **Per-run persistence.** No middleware is injected because the user has provided a custom chat client stack. Messages are persisted at the end of the run. |
87-
| `true` | `true` | **User responsibility.** The system checks whether the custom chat client stack includes a `ServiceStoredSimulatingChatClient`. If not, a warning is emitted — the user is expected to have added their own per-service-call persistence mechanism. End-of-run persistence is skipped. |
87+
| `true` | `true` | **User responsibility.** The system checks whether the custom chat client stack includes a `PerServiceCallChatHistoryPersistingChatClient`. If not, a warning is emitted — the user is expected to have added their own per-service-call persistence mechanism. End-of-run persistence is skipped. |
8888

8989
### Consequences
9090

9191
- Good, because per-run persistence is atomic by default — chat history is only updated when the full run succeeds, satisfying driver B.
9292
- Good, because the default mental model is simple: one run = one history update, satisfying driver D.
93-
- Good, because users who opt into `SimulateServiceStoredChatHistory` get stored history that matches the service's behavior for both timing and content, fully satisfying driver A.
93+
- Good, because users who opt into `RequirePerServiceCallChatHistoryPersistence` get stored history that matches the service's behavior for both timing and content, fully satisfying driver A.
9494
- Good, because per-service-call persistence preserves intermediate progress if the process is interrupted, satisfying driver C when opted in.
9595
- Good, because no separate `FunctionResultContent` trimming logic is needed when per-service-call persistence is active — it is naturally handled.
9696
- Good, because conflict detection (configurable via `ThrowOnChatHistoryProviderConflict`, `WarnOnChatHistoryProviderConflict`, `ClearOnChatHistoryProviderConflict`) prevents misconfiguration when a service returns a `ConversationId` alongside a configured `ChatHistoryProvider`.
9797
- Bad, because per-service-call persistence (when opted in) may leave chat history in an incomplete state if the run fails mid-loop (e.g., `FunctionCallContent` stored without corresponding `FunctionResultContent`), requiring manual recovery in rare cases.
98-
- Neutral, because users who want per-service-call consistency can opt in via `SimulateServiceStoredChatHistory = true`, satisfying driver E.
98+
- Neutral, because users who want per-service-call consistency can opt in via `RequirePerServiceCallChatHistoryPersistence = true`, satisfying driver E.
9999
- Neutral, because increased write frequency from per-service-call persistence may impact performance for some storage backends; this can be mitigated with a caching decorator.
100100

101101
### Implementation Notes
102102

103103
#### Conversation ID Consistency
104104

105-
We should introduce a separate `ConversationIdPersistingChatClient`, middleware which allows us to
106-
persist response `ConversationIds` during the FICC loop. This could be used with or without
107-
`ServiceStoredSimulatingChatClient`.
105+
When `RequirePerServiceCallChatHistoryPersistence` is enabled, the `PerServiceCallChatHistoryPersistingChatClient`
106+
decorator also updates `session.ConversationId` after each service call. This handles two scenarios:
107+
108+
1. **Framework-managed chat history** — the decorator sets a sentinel `ConversationId` on the response
109+
so that `FunctionInvokingChatClient` treats the conversation as service-managed (clearing accumulated
110+
history between iterations and not injecting duplicate `FunctionCallContent` during approval processing).
111+
112+
2. **Service-stored chat history** — when the service returns a real `ConversationId`, the decorator
113+
updates `session.ConversationId` immediately after each service call, rather than deferring the update
114+
to the end of the run. This ensures intermediate ConversationId changes are captured even if the
115+
process is interrupted mid-loop.
116+
117+
For some service-stored scenarios (e.g., the Conversations API with the Responses API), there is only
118+
one thread with one ID, so every service call returns the same ConversationId and this per-call update
119+
makes no practical difference. Enabling `RequirePerServiceCallChatHistoryPersistence` ensures consistent
120+
per-service-call behavior across all service types regardless of how they manage ConversationIds.
108121

‎dotnet/samples/02-agents/Agents/Agent_Step19_InFunctionLoopCheckpointing/Program.cs‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
// Copyright (c) Microsoft. All rights reserved.
22

33
// This sample demonstrates how the ChatClientAgent persists chat history after each individual
4-
// call to the AI service, using the SimulateServiceStoredChatHistory option.
4+
// call to the AI service, using the RequirePerServiceCallChatHistoryPersistence option.
55
// When an agent uses tools, FunctionInvokingChatClient may loop multiple times
66
// (service call → tool execution → service call), and intermediate messages (tool calls and
77
// results) are persisted after each service call. This allows you to inspect or recover them
88
// even if the process is interrupted mid-loop, but may also result in chat history that is not
99
// yet finalized (e.g., tool calls without results) being persisted, which may be undesirable in some cases.
1010
//
1111
// To use end-of-run persistence instead (atomic run semantics), remove the
12-
// SimulateServiceStoredChatHistory = true setting (or set it to false). End-of-run
12+
// RequirePerServiceCallChatHistoryPersistence = true setting (or set it to false). End-of-run
1313
// persistence is the default behavior.
1414
//
1515
// The sample runs two multi-turn conversations: one using non-streaming (RunAsync) and one
@@ -54,7 +54,7 @@ static string GetTime([Description("The city name.")] string city) =>
5454
_ => $"{city}: time data not available."
5555
};
5656

57-
// Create the agent — per-service-call persistence is enabled via SimulateServiceStoredChatHistory.
57+
// Create the agent — per-service-call persistence is enabled via RequirePerServiceCallChatHistoryPersistence.
5858
// The in-memory ChatHistoryProvider is used by default when the service does not require service stored chat
5959
// history, so for those cases, we can inspect the chat history via session.TryGetInMemoryChatHistory().
6060
IChatClient chatClient = string.Equals(store, "TRUE", StringComparison.OrdinalIgnoreCase) ?
@@ -64,7 +64,7 @@ static string GetTime([Description("The city name.")] string city) =>
6464
new ChatClientAgentOptions
6565
{
6666
Name = "WeatherAssistant",
67-
SimulateServiceStoredChatHistory = true,
67+
RequirePerServiceCallChatHistoryPersistence = true,
6868
ChatOptions = new()
6969
{
7070
Instructions = "You are a helpful assistant. When asked about multiple cities, call the appropriate tool for each city.",

‎dotnet/samples/02-agents/Agents/Agent_Step19_InFunctionLoopCheckpointing/README.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,19 @@
11
# In-Function-Loop Checkpointing
22

3-
This sample demonstrates how `ChatClientAgent` can persist chat history after each individual call to the AI service using the `SimulateServiceStoredChatHistory` option. This per-service-call persistence ensures intermediate progress is saved during the function invocation loop.
3+
This sample demonstrates how `ChatClientAgent` can persist chat history after each individual call to the AI service using the `RequirePerServiceCallChatHistoryPersistence` option. This per-service-call persistence ensures intermediate progress is saved during the function invocation loop.
44

55
## What This Sample Shows
66

7-
When an agent uses tools, the `FunctionInvokingChatClient` loops multiple times (service call → tool execution → service call → …). By enabling `SimulateServiceStoredChatHistory = true`, chat history is persisted after each service call via the `ServiceStoredSimulatingChatClient` decorator:
7+
When an agent uses tools, the `FunctionInvokingChatClient` loops multiple times (service call → tool execution → service call → …). By enabling `RequirePerServiceCallChatHistoryPersistence = true`, chat history is persisted after each service call via the `PerServiceCallChatHistoryPersistingChatClient` decorator:
88

9-
- A `ServiceStoredSimulatingChatClient` decorator is inserted into the chat client pipeline
9+
- A `PerServiceCallChatHistoryPersistingChatClient` decorator is inserted into the chat client pipeline
1010
- Before each service call, the decorator loads history from the `ChatHistoryProvider` and prepends it to the request
1111
- After each service call, the decorator notifies the `ChatHistoryProvider` (and any `AIContextProvider` instances) with the new messages
1212
- Only **new** messages are sent to providers on each notification — messages that were already persisted in an earlier call within the same run are deduplicated automatically
1313

14-
By default (without `SimulateServiceStoredChatHistory`), chat history is persisted at the end of the full agent run instead. To use per-service-call persistence, set `SimulateServiceStoredChatHistory = true` on `ChatClientAgentOptions`.
14+
By default (without `RequirePerServiceCallChatHistoryPersistence`), chat history is persisted at the end of the full agent run instead. To use per-service-call persistence, set `RequirePerServiceCallChatHistoryPersistence = true` on `ChatClientAgentOptions`.
1515

16-
With `SimulateServiceStoredChatHistory` = true, the behavior matches that of chat history stored in the underlying AI service exactly.
16+
With `RequirePerServiceCallChatHistoryPersistence` = true, the behavior matches that of chat history stored in the underlying AI service exactly.
1717

1818
Per-service-call persistence is useful for:
1919
- **Crash recovery** — if the process is interrupted mid-loop, the intermediate tool calls and results are already persisted
@@ -29,7 +29,7 @@ The sample asks the agent about the weather and time in three cities. The model
2929
```
3030
ChatClientAgent
3131
└─ FunctionInvokingChatClient (handles tool call loop)
32-
└─ ServiceStoredSimulatingChatClient (persists after each service call)
32+
└─ PerServiceCallChatHistoryPersistingChatClient (persists after each service call)
3333
└─ Leaf IChatClient (Azure OpenAI)
3434
```
3535

0 commit comments

Comments
 (0)