braintrust

Braintrust Go SDK

Overview

This library provides tools for evaluating and tracing AI applications in Braintrust. Use it to:

• Evaluate your AI models with custom test cases and scoring functions
• Trace LLM calls and monitor AI application performance with OpenTelemetry
• Integrate seamlessly with OpenAI, Anthropic, Google Gemini, Genkit, ADK, CloudWeGo Eino, LangChainGo, and other LLM providers

This SDK is currently in BETA status and APIs may change.

Installation

go get github.com/braintrustdata/braintrust-sdk-go

export BRAINTRUST_API_KEY="your-api-key"  # Get from https://www.braintrust.dev/app/settings

Each tracing integration is published as its own Go module. Install only the ones you need:

go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai       # OpenAI (openai-go)
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/anthropic     # Anthropic
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/genai         # Google GenAI
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/genkit        # Firebase Genkit
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/adk           # Google ADK
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/cloudwego/eino # CloudWeGo Eino
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/langchaingo   # LangChainGo
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/github.com/sashabaranov/go-openai # sashabaranov/go-openai

Or install all integrations at once with the meta-module:

go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/all

Instrumentation

Trace LLM calls with automatic or manual instrumentation.

Automatic Instrumentation

Use Orchestrion to automatically inject tracing at compile time—no code changes required.

1. Install orchestrion:

go install github.com/DataDog/orchestrion@v1.6.1

2. Create orchestrion.tool.go in your project root:

//go:build tools

package main

import (
    _ "github.com/DataDog/orchestrion"
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/all" // Dedicated meta-module for all Braintrust LLM integrations
)

Or import only the integrations you need:

import (
    _ "github.com/DataDog/orchestrion"
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai"    // OpenAI (openai-go)
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/anthropic" // Anthropic
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/genai"     // Google GenAI
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/genkit"    // Firebase Genkit
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/adk"       // Google ADK
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/cloudwego/eino" // CloudWeGo Eino
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/langchaingo" // LangChainGo
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/github.com/sashabaranov/go-openai" // sashabaranov/go-openai
)

3. Build with orchestrion:

# Build with orchestrion
orchestrion go build ./...

# Or configure GOFLAGS to use orchestrion automatically
export GOFLAGS="-toolexec='orchestrion toolexec'"
go build ./...

4. Initialize OpenTelemetry and Braintrust in your application:

import (
    "context"
    "log"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/sdk/trace"
    "github.com/braintrustdata/braintrust-sdk-go"
)

func main() {
    ctx := context.Background()

    // Set up OpenTelemetry tracer
    tp := trace.NewTracerProvider()
    defer tp.Shutdown(ctx)
    otel.SetTracerProvider(tp)

    // Initialize Braintrust (registers the exporter)
    _, err := braintrust.New(tp, braintrust.WithProject("my-project"))
    if err != nil {
        log.Fatal(err)
    }

    // Your LLM calls are now automatically traced
}

That's it! Your LLM client calls are now automatically traced. No middleware or wrapper code needed in your application.

Manual Instrumentation

If you prefer explicit control, you can add tracing middleware manually to your LLM clients. See the Manual Instrumentation Guide for detailed examples with OpenAI, Anthropic, Google Gemini, and other providers.

Evaluations

Run evals with custom test cases and scoring functions:

package main

import (
    "context"
    "log"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/sdk/trace"

    "github.com/braintrustdata/braintrust-sdk-go"
    "github.com/braintrustdata/braintrust-sdk-go/eval"
)

func main() {
    ctx := context.Background()

    // Set up OpenTelemetry tracer
    tp := trace.NewTracerProvider()
    defer tp.Shutdown(ctx)
    otel.SetTracerProvider(tp)

    // Initialize Braintrust
    client, err := braintrust.New(tp)
    if err != nil {
        log.Fatal(err)
    }

    // Create an evaluator with your task's input and output types
    evaluator := braintrust.NewEvaluator[string, string](client)

    // Run an evaluation
    _, err = evaluator.Run(ctx, eval.Opts[string, string]{
        Experiment: "greeting-experiment",
        Dataset: eval.NewDataset([]eval.Case[string, string]{
            {Input: "World", Expected: "Hello World"},
            {Input: "Alice", Expected: "Hello Alice"},
        }),
        Task: eval.T(func(ctx context.Context, input string) (string, error) {
            return "Hello " + input, nil
        }),
        Scorers: []eval.Scorer[string, string]{
            eval.NewScorer("exact_match", func(ctx context.Context, r eval.TaskResult[string, string]) (eval.Scores, error) {
                score := 0.0
                if r.Expected == r.Output {
                    score = 1.0
                }
                return eval.S(score), nil
            }),
        },
    })
    if err != nil {
        log.Fatal(err)
    }
}

API Client

Manage Braintrust resources programmatically:

package main

import (
    "context"
    "log"

    "go.opentelemetry.io/otel/sdk/trace"

    "github.com/braintrustdata/braintrust-sdk-go"
    functionsapi "github.com/braintrustdata/braintrust-sdk-go/api/functions"
)

func main() {
    ctx := context.Background()

    // Create tracer provider
    tp := trace.NewTracerProvider()
    defer tp.Shutdown(ctx)

    // Initialize Braintrust
    client, err := braintrust.New(tp,
        braintrust.WithProject("my-project"),
    )
    if err != nil {
        log.Fatal(err)
    }

    // Get API client
    api := client.API()

    // Create a prompt
    prompt, err := api.Functions().Create(ctx, functionsapi.CreateParams{
        ProjectID: "your-project-id",
        Name:      "My Prompt",
        Slug:      "my-prompt",
        FunctionData: map[string]any{
            "type": "prompt",
        },
        PromptData: map[string]any{
            "prompt": map[string]any{
                "type": "chat",
                "messages": []map[string]any{
                    {
                        "role":    "system",
                        "content": "You are a helpful assistant.",
                    },
                    {
                        "role":    "user",
                        "content": "{{input}}",
                    },
                },
            },
            "options": map[string]any{
                "model": "gpt-4o-mini",
            },
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    _ = prompt // Prompt is ready to use
}

Examples

Complete working examples are available in examples/:

Getting Started:

• openai - OpenAI tracing
• anthropic - Anthropic tracing
• genai - Google Gemini tracing
• genkit - Firebase Genkit middleware tracing

Evaluations:

• evals - Evaluations with custom scorers
• datasets - Run evals against downloaded datasets
• dataset-api - Create datasets, use prompts, run evals
• scorers - Custom scoring with online and code-based scorers

Alternative Providers & Libraries:

• sashabaranov-openai - sashabaranov/go-openai tracing
• openrouter - OpenRouter tracing
• langchaingo - LangChainGo integration
• adk - Google ADK agent tracing
• cloudwego/eino - CloudWeGo Eino integration

Advanced:

• manual-llm-logging - Manually log LLM calls
• attachments - Include images and files in traces
• prompts - Use Braintrust hosted prompts
• distributed-tracing - W3C baggage propagation across services
• otel - Add Braintrust to existing OpenTelemetry setup

Features

• Evaluations - Systematic testing with custom scoring functions
• Tracing - Automatic instrumentation for major LLM providers
• Datasets - Manage and version evaluation datasets
• Experiments - Track versions and configurations
• Observability - Monitor AI applications in production

Documentation

• Braintrust Documentation
• API Reference

Contributing

See CONTRIBUTING.md for development setup and contribution guidelines.

License

Apache License 2.0. See LICENSE for details.