OpenAI SDK | Respan Docs

The OpenAI SDK is the official client for OpenAI’s APIs, available for both Python and TypeScript/JavaScript. It supports Chat Completions and the Responses API. Respan gives you full observability over every OpenAI call, streamed response, and tool invocation — and gateway routing to 250+ models with prompt management.

Set up Respan

Create an account at platform.respan.ai and grab an API key. For gateway, also add credits or a provider key.

Run npx @respan/cli setup to set up with your coding agent.

Example projects

Tracing

Gateway

Setup

Install packages

$ pip install respan-ai respan-instrumentation-openai openai

Set environment variables

$ export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"
$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

OPENAI_API_KEY is used for OpenAI requests. RESPAN_API_KEY is used to export traces to Respan.

Initialize and run

1 from openai import OpenAI
2 from respan import Respan
3 from respan_instrumentation_openai import OpenAIInstrumentor
4 
5 respan = Respan(instrumentations=[OpenAIInstrumentor()])
6 
7 client = OpenAI()
8 
9 response = client.chat.completions.create(
10     model="gpt-4.1-nano",
11     messages=[{"role": "user", "content": "Say hello in three languages."}],
12 )
13 print(response.choices[0].message.content)
14 respan.flush()

View your trace

Open the Traces page to see your auto-instrumented LLM spans.

Configuration

Parameter	Type	Default	Description
`api_key`	`str \| None`	`None`	Falls back to `RESPAN_API_KEY` env var.
`base_url`	`str \| None`	`None`	Falls back to `RESPAN_BASE_URL` env var.
`instrumentations`	`list`	`[]`	Plugin instrumentations to activate (e.g. `OpenAIInstrumentor()`).
`customer_identifier`	`str \| None`	`None`	Default customer identifier for all spans.
`metadata`	`dict \| None`	`None`	Default metadata attached to all spans.
`environment`	`str \| None`	`None`	Environment tag (e.g. `"production"`).

Attributes

In Respan()

Set defaults at initialization — these apply to all spans.

1 from respan import Respan
2 from respan_instrumentation_openai import OpenAIInstrumentor
3 
4 respan = Respan(
5     instrumentations=[OpenAIInstrumentor()],
6     customer_identifier="user_123",
7     metadata={"service": "chat-api", "version": "1.0.0"},
8 )

With propagate_attributes

Override per-request using a context scope.

1 from openai import OpenAI
2 from respan import Respan, propagate_attributes
3 from respan_instrumentation_openai import OpenAIInstrumentor
4 
5 respan = Respan(instrumentations=[OpenAIInstrumentor()])
6 client = OpenAI()
7 
8 def handle_request(user_id: str, question: str):
9     with propagate_attributes(
10         customer_identifier=user_id,
11         thread_identifier="conv_abc_123",
12         metadata={"plan": "pro"},
13     ):
14         response = client.chat.completions.create(
15             model="gpt-4.1-nano",
16             messages=[{"role": "user", "content": question}],
17         )
18         print(response.choices[0].message.content)

Attribute	Type	Description
`customer_identifier`	`str`	Identifies the end user in Respan analytics.
`thread_identifier`	`str`	Groups related messages into a conversation.
`metadata`	`dict`	Custom key-value pairs. Merged with default metadata.

Decorators (optional)

Decorators are not required. All OpenAI calls are auto-traced by the instrumentor. Use @workflow and @task (Python) or withWorkflow and withTask (TypeScript) to add structure when you want to group related calls into a named workflow with nested tasks.

1 from openai import OpenAI
2 from respan import Respan, workflow, task
3 from respan_instrumentation_openai import OpenAIInstrumentor
4 
5 respan = Respan(instrumentations=[OpenAIInstrumentor()])
6 client = OpenAI()
7 
8 @task(name="generate_outline")
9 def outline(topic: str) -> str:
10     response = client.chat.completions.create(
11         model="gpt-4.1-nano",
12         messages=[
13             {"role": "system", "content": "Create a brief outline."},
14             {"role": "user", "content": topic},
15         ],
16     )
17     return response.choices[0].message.content
18 
19 @workflow(name="content_pipeline")
20 def pipeline(topic: str):
21     plan = outline(topic)
22     response = client.chat.completions.create(
23         model="gpt-4.1-nano",
24         messages=[
25             {"role": "system", "content": "Write content from this outline."},
26             {"role": "user", "content": plan},
27         ],
28     )
29     print(response.choices[0].message.content)
30 
31 pipeline("Benefits of API gateways")
32 respan.flush()

Examples

Streaming

Streaming responses are auto-traced like regular completions.

1 stream = client.chat.completions.create(
2     model="gpt-4.1-nano",
3     messages=[{"role": "user", "content": "Write a haiku about Python."}],
4     stream=True,
5 )
6 
7 for chunk in stream:
8     content = chunk.choices[0].delta.content
9     if content:
10         print(content, end="", flush=True)

Tool calls

Function calling is auto-traced. Wrap the workflow with @workflow and @task decorators for a structured trace tree.

1 import json
2 from openai import OpenAI
3 from respan import Respan, workflow, task
4 from respan_instrumentation_openai import OpenAIInstrumentor
5 
6 respan = Respan(instrumentations=[OpenAIInstrumentor()])
7 client = OpenAI()
8 
9 tools = [
10     {
11         "type": "function",
12         "function": {
13             "name": "get_weather",
14             "description": "Get the weather for a city.",
15             "parameters": {
16                 "type": "object",
17                 "properties": {"city": {"type": "string"}},
18                 "required": ["city"],
19             },
20         },
21     }
22 ]
23 
24 @task(name="get_weather")
25 def get_weather(city: str) -> str:
26     return f"Sunny, 72F in {city}"
27 
28 @workflow(name="weather_assistant")
29 def run(question: str):
30     messages = [{"role": "user", "content": question}]
31 
32     response = client.chat.completions.create(
33         model="gpt-4.1-nano",
34         messages=messages,
35         tools=tools,
36     )
37     message = response.choices[0].message
38 
39     if message.tool_calls:
40         messages.append(message)
41         for tc in message.tool_calls:
42             args = json.loads(tc.function.arguments)
43             result = get_weather(**args)
44             messages.append(
45                 {"role": "tool", "tool_call_id": tc.id, "content": result}
46             )
47 
48         final = client.chat.completions.create(
49             model="gpt-4.1-nano",
50             messages=messages,
51             tools=tools,
52         )
53         print(f"Answer: {final.choices[0].message.content}")
54 
55 run("What's the weather in Paris?")
56 respan.flush()

Structured output

JSON mode with Pydantic models is auto-traced.

1 from pydantic import BaseModel
2 from openai import OpenAI
3 
4 client = OpenAI()
5 
6 class MovieReview(BaseModel):
7     title: str
8     rating: int
9     summary: str
10     pros: list[str]
11     cons: list[str]
12 
13 response = client.beta.chat.completions.parse(
14     model="gpt-4.1-nano",
15     messages=[
16         {"role": "system", "content": "You are a film critic. Rate movies 1-10."},
17         {"role": "user", "content": "Review: The Matrix"},
18     ],
19     response_format=MovieReview,
20 )
21 result = response.choices[0].message.parsed
22 print(f"{result.title} - {result.rating}/10")