Skip to content
This repository was archived by the owner on Apr 30, 2026. It is now read-only.

Commit 06f884a

Browse files
thucpnthucpn
authored
feat: return structured object from llm.exec (#2198)
1 parent 9e66861 commit 06f884a

16 files changed

Lines changed: 538 additions & 66 deletions

File tree

‎.changeset/five-peaches-brush.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@llamaindex/core": patch
3+
---
4+
5+
feat: return structured object from llm.exec

‎.changeset/red-bushes-brake.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@llamaindex/workflow": patch
3+
---
4+
5+
feat: return structured data when using agent.run()

‎apps/next/src/content/docs/llamaindex/modules/agents/agent_workflow.mdx‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,58 @@ console.log(result.data.result); // Baby Llama is called cria
3737
console.log(result.data.message); // { role: 'assistant', content: 'Baby Llama is called cria' }
3838
```
3939

40+
### Structured Output
41+
42+
You can extract structured data from agent responses by providing a `responseFormat` with a Zod schema. This is useful when you need the agent's response in a specific format for further processing:
43+
44+
```typescript
45+
import { z } from "zod";
46+
import { tool } from "llamaindex";
47+
import { agent } from "@llamaindex/workflow";
48+
import { openai } from "@llamaindex/openai";
49+
50+
// Define a weather tool
51+
const weatherTool = tool({
52+
name: "weatherTool",
53+
description: "Get weather information",
54+
parameters: z.object({
55+
location: z.string(),
56+
}),
57+
execute: ({ location }) => {
58+
return `The weather in ${location} is sunny. The temperature is 72 degrees. The humidity is 50%. The wind speed is 10 mph.`;
59+
},
60+
});
61+
62+
// Define the structure you want for the response
63+
const responseSchema = z.object({
64+
temperature: z.number(),
65+
humidity: z.number(),
66+
windSpeed: z.number(),
67+
});
68+
69+
// Create the agent
70+
const weatherAgent = agent({
71+
name: "weatherAgent",
72+
tools: [weatherTool],
73+
llm: openai({ model: "gpt-4.1-mini" }),
74+
});
75+
76+
// Run with structured output
77+
const result = await weatherAgent.run("What's the weather in Tokyo?", {
78+
responseFormat: responseSchema,
79+
});
80+
81+
console.log("Natural language result:", result.data.result);
82+
console.log("Structured data:", result.data.object);
83+
// Output: { temperature: 72, humidity: 50, windSpeed: 10 }
84+
```
85+
86+
The agent will:
87+
1. Use the weather tool to get the raw weather information
88+
2. Process that information through the LLM
89+
3. Extract structured data according to your schema
90+
4. Return both the natural language response and the structured object
91+
4092
### Event Streaming
4193

4294
Agent Workflows provide a unified interface for event streaming, making it easy to track and respond to different events during execution:

‎apps/next/src/content/docs/llamaindex/modules/agents/low-level.mdx‎

Lines changed: 36 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ Sometimes your need more control over LLM interactions than what high-level agen
99
Use `llm.exec` when you need to:
1010
- Build custom agent logic in [workflow](/docs/llamaindex/modules/agents/workflows) steps
1111
- Have precise control over message handling and tool execution
12+
- Extract structured data from LLM responses
1213

1314
## Basic Usage
1415

@@ -51,6 +52,38 @@ messages.push(...newMessages);
5152

5253
> `newMessages` is an array as each tool call generates two messages: a tool call message and the tool call result message.
5354
55+
## Structured Output
56+
57+
You can use `responseFormat` with a Zod schema to get structured data from the LLM response:
58+
59+
```ts
60+
import { openai } from "@llamaindex/openai";
61+
import { ChatMessage } from "llamaindex";
62+
import z from "zod";
63+
64+
const llm = openai({ model: "gpt-4.1-mini" });
65+
66+
const schema = z.object({
67+
title: z.string(),
68+
author: z.string(),
69+
year: z.number(),
70+
});
71+
72+
const messages = [
73+
{
74+
role: "user",
75+
content: "I have been reading La Divina Commedia by Dante Alighieri, published in 1321",
76+
} as ChatMessage,
77+
];
78+
79+
const { newMessages, toolCalls, object } = await llm.exec({
80+
messages,
81+
responseFormat: schema,
82+
});
83+
84+
console.log(object); // { title: "La Divina Commedia", author: "Dante Alighieri", year: 1321 }
85+
```
86+
5487
## Agent Loop Pattern
5588

5689
A common pattern is to use `llm.exec` in a loop until the LLM stops making tool calls:
@@ -102,7 +135,7 @@ For real-time responses, use the `stream` option to get the assistant's response
102135

103136
```ts
104137
import { openai } from "@llamaindex/openai";
105-
import { tool } from "llamaindex";
138+
import { ChatMessage, tool } from "llamaindex";
106139
import z from "zod";
107140

108141
async function streamingAgentLoop() {
@@ -153,6 +186,7 @@ async function streamingAgentLoop() {
153186

154187
- **`newMessages`**: Array of new chat messages including the LLM response and any tool call messages (call or result). This is a function return the array when streaming.
155188
- **`toolCalls`**: Array of tool calls made by the LLM
189+
- **`object`**: The structured object when using `responseFormat` with a Zod schema (undefined if no schema is provided)
156190
- **`stream`**: Async iterable for streaming responses (only when `stream: true`)
157191

158192
## Best Practices
@@ -161,4 +195,4 @@ For using `llm.exec` in an agent loop, take care to:
161195

162196
1. **Maintain message history**: Always add `newMessages` to your conversation history
163197
2. **Set exit conditions**: Implement proper logic to avoid infinite loops
164-
198+
3. **Handle structured output**: When using `responseFormat`, the `object` property contains your parsed data

‎examples/agents/agent/structured-object.ts‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
import { z } from "zod";
2+
3+
import { openai } from "@llamaindex/openai";
4+
import { agent } from "@llamaindex/workflow";
5+
import { tool } from "llamaindex";
6+
7+
const weatherTool = tool({
8+
name: "weatherTool",
9+
description: "Get weather information",
10+
parameters: z.object({
11+
location: z.string(),
12+
}),
13+
execute: ({ location }) => {
14+
return `The weather in ${location} is sunny. The temperature is 72 degrees. The humidity is 50%. The wind speed is 10 mph.`;
15+
},
16+
});
17+
18+
const responseSchema = z.object({
19+
temperature: z.number(),
20+
humidity: z.number(),
21+
windSpeed: z.number(),
22+
});
23+
24+
const myAgent = agent({
25+
name: "myAgent",
26+
tools: [weatherTool],
27+
llm: openai({ model: "gpt-4.1-mini" }),
28+
});
29+
30+
async function main() {
31+
const result = await myAgent.run("What's the weather in Tokyo?", {
32+
responseFormat: responseSchema,
33+
});
34+
35+
console.log("result.data.result: ", result.data.result);
36+
console.log("result.data.object: ", result.data.object);
37+
}
38+
39+
main().catch(console.error);

‎examples/agents/tools/exec-object.ts‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
import { openai } from "@llamaindex/openai";
2+
import { ChatMessage } from "llamaindex";
3+
import z from "zod";
4+
5+
const llm = openai({ model: "gpt-4.1-mini" });
6+
7+
const schema = z.object({
8+
title: z.string(),
9+
author: z.string(),
10+
year: z.number(),
11+
});
12+
13+
const messages: ChatMessage[] = [
14+
{
15+
role: "user",
16+
content: `I have been reading La Divina Commedia by Dante Alighieri, published in 1321`,
17+
},
18+
];
19+
20+
async function main() {
21+
{
22+
// Non-streaming
23+
const { object } = await llm.exec({ messages, responseFormat: schema });
24+
console.log("Non-streaming object:", object);
25+
}
26+
27+
{
28+
// Streaming
29+
let exit = false;
30+
do {
31+
const { stream, newMessages, toolCalls, object } = await llm.exec({
32+
messages,
33+
stream: true,
34+
responseFormat: schema,
35+
});
36+
37+
for await (const chunk of stream) {
38+
console.log(chunk.delta);
39+
}
40+
console.log("Streaming object:", object);
41+
42+
messages.push(...newMessages());
43+
exit = toolCalls.length === 0;
44+
} while (!exit);
45+
}
46+
}
47+
48+
main().catch(console.error);

‎examples/agents/tools/response-format-exec.ts‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ const responseSchema = z.object({
1313
async function main() {
1414
const messages: ChatMessage[] = [];
1515
let toolCalls: ToolCall[] = [];
16+
let object: z.infer<typeof responseSchema> | undefined;
1617
do {
1718
const result = await llm.exec({
1819
messages: [
@@ -27,13 +28,14 @@ async function main() {
2728
],
2829
responseFormat: responseSchema,
2930
});
30-
console.log(result.newMessages[0].content);
31+
object = result.object;
3132
messages.push(...result.newMessages);
3233
toolCalls = result.toolCalls;
3334
} while (toolCalls.length == 0);
3435

35-
console.log(messages[1].content);
36+
console.log(messages);
3637
console.log(toolCalls);
38+
console.log(object);
3739
}
3840

3941
main().catch(console.error);

0 commit comments

Comments
 (0)