Pinned
Topics is now GA on all plans.
Continuously find the patterns worth investigating across your production traffic.
00:00
Braintrust
732 posts
Braintrust
@braintrust
The observability layer for production AI.
Joined August 2023
- Traces tell you how customers are using your agents. Topics groups those interactions into patterns so you can uncover opportunities for improvement. Braintrust is hosting a workshop on using Topics to identify customer use cases from production data and turning those
- Build a full eval pipeline (dataset, prompt, scorer, experiment) using just the Braintrust CLI and skills. In this video, we test GPT-5 on chess puzzles and analyze the results of our experiments with only natural language, no code written. Watch here → braintrustdata.link/CLI-skills-vid…
00:00 - The inaugural Agent Open is happening June 30 in SF. Come talk about AI observability, then show off your skills at one thing agents can’t do: pickleball. Hosted by the teams at @braintrust, @cursor_ai, @llama_index, @turbopuffer, @p0, @modal,+ @browserbase.
- What's new: -Topics is now GA, with $249 in credits for Pro plans -Multi-user human review, with averaged scores -Workload identity federation for Anthropic, Vertex AI, and Azure -Run remote evals and sandboxes as experiments -Automate data preparation with dataset pipelines
00:00 - Replying to @braintrustIt sounds simple, but the hard part is making it work across millions of production traces without blowing out token costs or breaking down at scale. Here's how we built Topics →
- Raw agent traces can include millions of tokens across hundreds of spans. Too large for direct embedding, too irregular for classic topic modeling, and too high-volume for full-trace LLM classification. Topics solves this by summarizing traces into facets, then continuously
- Production traces capture where your AI falls short and what users are trying to do. Building evals with that data is how you catch failures earlier and decide what to ship next. Braintrust is leading a workshop on how to: - Use the patterns Braintrust surfaces automatically -
