Creating persistent chat sessions

JavaScript Go

Copy as Markdown Copied!

Beta

This feature of Genkit is in Beta, which means it is not yet part of Genkit’s stable API. APIs of beta features may change in minor version releases.

Many of your users will have interacted with large language models for the first time through chatbots. Although LLMs are capable of much more than simulating conversations, it remains a familiar and useful style of interaction. Even when your users will not be interacting directly with the model in this way, the conversational style of prompting is a powerful way to influence the output generated by an AI model.

To support this style of interaction, Genkit provides a set of interfaces and abstractions that make it easier for you to build chat-based LLM applications.

Before you begin

Before reading this page, you should be familiar with the content covered on the Generating content with AI models page.

If you want to run the code examples on this page, first complete the steps in the Getting started guide. All of the examples assume that you have already installed Genkit as a dependency in your project.

Note that the chat API is currently in beta and must be used from the genkit/beta package.

Chat session basics

Genkit by Example: Simple Chatbot View a live example of a simple chatbot built with Genkit.

Here is a minimal, console-based, chatbot application:

import { genkit } from 'genkit/beta';
import { googleAI } from '@genkit-ai/google-genai';

import { createInterface } from 'node:readline/promises';

const ai = genkit({
  plugins: [googleAI()],
  model: googleAI.model('gemini-2.5-flash'),
});

async function main() {
  const chat = ai.chat();
  console.log("You're chatting with Gemini. Ctrl-C to quit.\n");
  const readline = createInterface(process.stdin, process.stdout);
  while (true) {
    const userInput = await readline.question('> ');
    const { text } = await chat.send(userInput);
    console.log(text);
  }
}

main();

A chat session with this program looks something like the following example:

You're chatting with Gemini. Ctrl-C to quit.

> hi
Hi there! How can I help you today?

> my name is pavel
Nice to meet you, Pavel! What can I do for you today?

> what's my name?
Your name is Pavel! I remembered it from our previous interaction.

Is there anything else I can help you with?

As you can see from this brief interaction, when you send a message to a chat session, the model can make use of the session so far in its responses. This is possible because Genkit does a few things behind the scenes:

• Retrieves the chat history, if any exists, from storage (more on persistence and storage later)
• Sends the request to the model, as with generate(), but automatically include the chat history
• Saves the model response into the chat history

Model configuration

The chat() method accepts most of the same configuration options as generate(). To pass configuration options to the model:

const chat = ai.chat({
  model: googleAI.model('gemini-2.5-flash'),
  system: "You're a pirate first mate. Address the user as Captain and assist " + 'them however you can.',
  config: {
    temperature: 1.3,
  },
});

Stateful chat sessions

In addition to persisting a chat session’s message history, you can also persist any arbitrary JavaScript object. Doing so can let you manage state in a more structured way then relying only on information in the message history.

To include state in a session, you need to instantiate a session explicitly:

interface MyState {
  userName: string;
}

const session = ai.createSession<MyState>({
  initialState: {
    userName: 'Pavel',
  },
});

You can then start a chat within the session:

const chat = session.chat();

To modify the session state based on how the chat unfolds, define tools and include them with your requests:

const changeUserName = ai.defineTool(
  {
    name: 'changeUserName',
    description: 'can be used to change user name',
    inputSchema: z.object({
      newUserName: z.string(),
    }),
  },
  async (input) => {
    await ai.currentSession<MyState>().updateState({
      userName: input.newUserName,
    });
    return `changed username to ${input.newUserName}`;
  },
);

const chat = session.chat({
  model: googleAI.model('gemini-2.5-flash'),
  tools: [changeUserName],
});
await chat.send('change user name to Kevin');

Multi-thread sessions

A single session can contain multiple chat threads. Each thread has its own message history, but they share a single session state.

const lawyerChat = session.chat('lawyerThread', {
  system: 'talk like a lawyer',
});
const pirateChat = session.chat('pirateThread', {
  system: 'talk like a pirate',
});

Session persistence (EXPERIMENTAL)

When you initialize a new chat or session, it’s configured by default to store the session in memory only. This is adequate when the session needs to persist only for the duration of a single invocation of your program, as in the sample chatbot from the beginning of this page. However, when integrating LLM chat into an application, you will usually deploy your content generation logic as stateless web API endpoints. For persistent chats to work under this setup, you will need to implement some kind of session storage that can persist state across invocations of your endpoints.

To add persistence to a chat session, you need to implement Genkit’s SessionStore interface. Here is an example implementation that saves session state to individual JSON files:

class JsonSessionStore<S = any> implements SessionStore<S> {
  async get(sessionId: string): Promise<SessionData<S> | undefined> {
    try {
      const s = await readFile(`${sessionId}.json`, { encoding: 'utf8' });
      const data = JSON.parse(s);
      return data;
    } catch {
      return undefined;
    }
  }

  async save(sessionId: string, sessionData: SessionData<S>): Promise<void> {
    const s = JSON.stringify(sessionData);
    await writeFile(`${sessionId}.json`, s, { encoding: 'utf8' });
  }
}

This implementation is probably not adequate for practical deployments, but it illustrates that a session storage implementation only needs to accomplish two tasks:

• Get a session object from storage using its session ID
• Save a given session object, indexed by its session ID

Once you’ve implemented the interface for your storage backend, pass an instance of your implementation to the session constructors:

// To create a new session:
const session = ai.createSession({
  store: new JsonSessionStore(),
});

// Save session.id so you can restore the session the next time the
// user makes a request.

// If the user has a session ID saved, load the session instead of creating
// a new one:
const session = await ai.loadSession(sessionId, {
  store: new JsonSessionStore(),
});

Next steps

• Learn about tool calling to add interactive capabilities to your chat sessions
• Explore context to understand how to pass information through chat sessions
• See developer tools for testing and debugging chat applications
• Check out generating content for understanding the underlying generation mechanics

Experimental

The sessions API in Genkit Go is experimental and may change in minor version releases.

Note

Genkit Go currently supports sessions for state management, but does not yet have a dedicated Chat API like JavaScript. You can build chat functionality by combining sessions with the generate API and managing message history in your session state.

Many of your users will have interacted with large language models for the first time through chatbots. Although LLMs are capable of much more than simulating conversations, it remains a familiar and useful style of interaction. Even when your users will not be interacting directly with the model in this way, the conversational style of prompting is a powerful way to influence the output generated by an AI model.

Genkit Go provides a sessions API that helps you manage stateful interactions with LLMs, persisting state across multiple requests.

Before you begin

Before reading this page, you should be familiar with the content covered on the Generating content with AI models page.

If you want to run the code examples on this page, first complete the steps in the Getting started guide. All of the examples assume that you have already installed Genkit as a dependency in your project.

The sessions API is in the experimental x package:

import "github.com/firebase/genkit/go/core/x/session"

Session basics

A session encapsulates a stateful execution environment with strongly-typed state that can be persisted across requests. Sessions are useful for maintaining user preferences, conversation context, or any application state that needs to survive between interactions.

Here’s a simple example that creates a session and uses it within a flow:

// Define your session state type
type ChatState struct {
    UserName string   `json:"userName"`
    History  []string `json:"history"`
}

// Create a store to persist sessions across requests
store := session.NewInMemoryStore[ChatState]()

genkit.DefineFlow(g, "chat", func(ctx context.Context, input string) (string, error) {
    sessionID := "user-123" // In practice, get this from the request

    // Load existing session or create new one
    sess, err := session.Load(ctx, store, sessionID)
    if err != nil {
        sess, err = session.New(ctx,
            session.WithID[ChatState](sessionID),
            session.WithStore(store),
            session.WithInitialState(ChatState{UserName: "Guest"}),
        )
        if err != nil {
            return "", err
        }
    }

    // Attach session to context for use in tools and prompts
    ctx = session.NewContext(ctx, sess)

    // Generate with the session-aware context
    return genkit.GenerateText(ctx, g,
        ai.WithModelName("googleai/gemini-2.5-flash"),
        ai.WithPrompt(input),
    )
})

Stateful sessions with tools

Sessions become powerful when combined with tools that can read and modify session state. This allows the model to maintain context and update state based on the conversation.

Here’s a shopping cart example that demonstrates stateful sessions:

// CartState holds the shopping cart items
type CartState struct {
    Items []string `json:"items"`
}

store := session.NewInMemoryStore[CartState]()
const sessionID = "shopping-session"

// Define a tool that adds items to the cart
addToCartTool := genkit.DefineTool(g, "addToCart",
    "Adds items to the shopping cart",
    func(ctx *ai.ToolContext, input struct{ Items []string }) ([]string, error) {
        sess := session.FromContext[CartState](ctx.Context)
        if sess == nil {
            return nil, fmt.Errorf("no session in context")
        }
        state := sess.State()
        state.Items = append(state.Items, input.Items...)
        if err := sess.UpdateState(ctx.Context, state); err != nil {
            return nil, err
        }
        return state.Items, nil
    },
)

// Define a tool that retrieves cart contents
getCartTool := genkit.DefineTool(g, "getCart",
    "Returns all items currently in the shopping cart",
    func(ctx *ai.ToolContext, input struct{}) ([]string, error) {
        sess := session.FromContext[CartState](ctx.Context)
        if sess == nil {
            return nil, fmt.Errorf("no session in context")
        }
        return sess.State().Items, nil
    },
)

genkit.DefineFlow(g, "manageCart", func(ctx context.Context, input string) (string, error) {
    sess, err := session.Load(ctx, store, sessionID)
    if err != nil {
        sess, err = session.New(ctx,
            session.WithID[CartState](sessionID),
            session.WithStore(store),
            session.WithInitialState(CartState{Items: []string{}}),
        )
        if err != nil {
            return "", err
        }
    }

    ctx = session.NewContext(ctx, sess)

    return genkit.GenerateText(ctx, g,
        ai.WithModelName("googleai/gemini-2.5-flash"),
        ai.WithSystem("You are a helpful shopping assistant. Use the provided tools to manage the user's cart."),
        ai.WithTools(addToCartTool, getCartTool),
        ai.WithPrompt(input),
    )
})

Using session state in prompts

When a session is attached to the context, the session state is automatically available in dotprompt templates using the {{@state.fieldName}} syntax. This allows you to personalize prompts based on the current session state without explicitly passing the state to the prompt.

For example, if your session state has a userName field:

type UserState struct {
    UserName    string   `json:"userName"`
    Preferences []string `json:"preferences"`
}

You can reference these fields directly in your dotprompt templates:

---
model: googleai/gemini-2.5-flash
---
Hello {{@state.userName}}! Based on your preferences ({{@state.preferences}}),
here are some recommendations for you.

{{prompt}}

When the prompt is rendered, {{@state.userName}} will be replaced with the actual value from the session state. This works automatically when you attach a session to the context using session.NewContext().

Session persistence

By default, sessions created without a store exist only in memory for the current request. To persist sessions across requests (even within the same process), you must provide a store. For production deployments, you’ll want to persist sessions to a database or other durable storage.

Implementing a custom store

To create a custom session store, implement the session.Store[S] interface:

type Store[S any] interface {
    // Get retrieves session data by ID. Returns nil if not found.
    Get(ctx context.Context, sessionID string) (*session.Data[S], error)
    // Save persists session data, creating or updating as needed.
    Save(ctx context.Context, sessionID string, data *session.Data[S]) error
}

Here’s an example JSON file-based store:

type JsonFileStore[S any] struct {
    dir string
}

func NewJsonFileStore[S any](dir string) *JsonFileStore[S] {
    return &JsonFileStore[S]{dir: dir}
}

func (s *JsonFileStore[S]) Get(ctx context.Context, sessionID string) (*session.Data[S], error) {
    path := filepath.Join(s.dir, sessionID+".json")
    data, err := os.ReadFile(path)
    if os.IsNotExist(err) {
        return nil, nil
    }
    if err != nil {
        return nil, err
    }

    var sessionData session.Data[S]
    if err := json.Unmarshal(data, &sessionData); err != nil {
        return nil, err
    }
    return &sessionData, nil
}

func (s *JsonFileStore[S]) Save(ctx context.Context, sessionID string, data *session.Data[S]) error {
    path := filepath.Join(s.dir, sessionID+".json")
    jsonData, err := json.Marshal(data)
    if err != nil {
        return err
    }
    return os.WriteFile(path, jsonData, 0644)
}

Firebase Firestore store

For production deployments, you can use the Firebase Firestore session store provided in the Firebase plugin:

import firebasex "github.com/firebase/genkit/go/plugins/firebase/x"

// Create a Firestore-backed session store
store, err := firebasex.NewFirestoreSessionStore[CartState](ctx, g,
    firebasex.WithCollection("sessions"),
    firebasex.WithTTL(24*time.Hour), // Sessions expire after 24 hours
)
if err != nil {
    log.Fatal(err)
}

// Use the store when creating sessions
sess, err := session.New(ctx,
    session.WithStore(store),
    session.WithInitialState(CartState{Items: []string{}}),
)

The Firestore store automatically handles:

• Persisting sessions across server restarts
• Session expiration via TTL
• Concurrent access from multiple server instances

Session API reference

Creating sessions

// Create an ephemeral session (not persisted, lives only for current request)
sess, err := session.New[MyState](ctx)

// Create with custom ID
sess, err := session.New(ctx, session.WithID[MyState]("my-session-id"))

// Create with initial state
sess, err := session.New(ctx, session.WithInitialState(MyState{Field: "value"}))

// Create with a store for persistence (required to load sessions later)
sess, err := session.New(ctx, session.WithStore(myStore))

// Combine options for a persistent session
sess, err := session.New(ctx,
    session.WithID[MyState]("my-session-id"),
    session.WithStore(myStore),
    session.WithInitialState(MyState{Field: "value"}),
)

Loading sessions

// Load an existing session from a store
sess, err := session.Load(ctx, store, sessionID)
if err != nil {
    // Handle error (session not found or store error)
}

Accessing session state

// Get the session ID
id := sess.ID()

// Get a copy of the current state
state := sess.State()

// Update the session state (persists to store)
state.Field = "new value"
err := sess.UpdateState(ctx, state)

Context integration

// Attach session to context
ctx = session.NewContext(ctx, sess)

// Retrieve session from context (in tools or other functions)
sess := session.FromContext[MyState](ctx)

// Get state without knowing the type (for template rendering)
state := session.StateFromContext(ctx)

Next steps

• Learn about tool calling to add interactive capabilities to your sessions
• Explore prompts to understand how to use session state in templates
• See flows for structuring your session-based applications

Feature unavailable for Python

Specialized Chat API for Genkit Python is under development.

Right now you can do it yourself using the generate API .