Fixing Turso Read-After-Write Consistency in Serverless

Fri, 15 Nov 2024 00:00:00 GMT

Turso is a powerful and versatile database platform for SQLite. One of its key features is the ability to easily deploy database replicas close to your users, which helps boost response times.

However, it can present challenges with web apps running in serverless setups, particularly with services like Vercel Edge and Cloudflare Workers, where unexpected data inconsistencies may arise if not managed properly.

In fact, if you’re using Turso with replicas, these inconsistencies may already be affecting your app without your knowledge. Want to know why? Keep reading to find out.

From Turso’s Docs

Writes and Propagation: All writes are forwarded to the primary database. Changes from the primary are eventually propagated to each replica, with a latency of 200-300ms.
Stale Data on New Connections: New connections may encounter stale data until a write operation updates the replication index. Replicas do not guarantee immediate reflection of primary’s changes, leading to potential data differences across replicas.

Source: Turso Docs

Why This Could be an Issue?

In traditional server environments, a continuous connection can be maintained with Turso, allowing immediate consistency for read-after-write operations.

However, in serverless setups like Vercel Edge or Cloudflare Workers, each time a function is called, it may create a new database connection. This stateless nature limits the ability of subsequent read operations to use Turso’s built-in replication index tracking, which relies on a continuous connection to ensure immediate consistency of data.

As a result, users, particularly those connecting from locations far from the primary database, may encounter stale data due to the time it takes for updates to reach all replicas.

While this delay is generally manageable, it can lead to significant challenges for critical operations like stock management or user authentication.

Example: Authentication

Imagine this scenario:

User successfully authenticate from Australia.
Session data is written to the primary database, located in the US.
Subsequent requests read session data from a new connection, using the closest database replica in Australia.
User session data may not be found due to replication latency (200-300ms). The data exists in the primary database, but not yet visible from the replica.
As a result, the user is mistakenly logged out.

How to Solve It?

To ensure read-after-write consistency for critical operations in serverless web apps, we can do the following:

Create a short-lived cookie: Upon completing a critical write operation, create a cookie with a short expiry (like 15 seconds). Attach this cookie to the user for whom you want to ensure read-after-write consistency.
Direct reads to Primary: If the cookie is present, force subsequent read operations to use the primary database. This ensures you’re accessing the most current data and bypassing potential latency issues from replicas.

Applying the Solution: Authentication with SvelteKit + Drizzle Example

In your SvelteKit action for handling logins, set a short-lived cookie immediately after creating a new user session in the database:

// +page.server.ts
export const actions = {
  login: async ({ cookies }) => {
    // ... login logic (create db session, etc.)

    // set a short-lived cookie after db session creation
    cookies.set('enforce_primarydb', 'true', { path: '/', maxAge: 15 });
  }
};

Implement a server hook to manage sessions and ensure that subsequent reads, like fetching the session from the database, use the primary database:

// +hooks.server.ts
import { getDatabase } from '$lib/server/database';

export const handle = async ({ event, resolve }) => {
  // check for the cookie to decide on primary db usage
  const usePrimaryDB = !!event.cookies.get('enforce_primarydb');
  const db = getDatabase({ usePrimaryDB });

  // ... verify session logic (read from db, etc.)

  return await resolve(event);
};

Ensure primary database usage with Drizzle:

// lib/server/database.ts
import { type Client, createClient } from '@libsql/client';
import { type LibSQLDatabase, drizzle } from 'drizzle-orm/libsql';
import * as schema from './schema';

let db: LibSQLDatabase<typeof schema> & { $client: Client };

const client = createClient({
  url: env.DATABASE_URL,
  authToken: env.DATABASE_AUTH_TOKEN
});
const clientPrimary = createClient({
  url: env.DATABASE_URL_PRIMARY,
  authToken: env.DATABASE_AUTH_TOKEN
});

export function getDatabase(ctx?: { usePrimaryDB?: boolean }) {
	if (ctx?.usePrimaryDB) db = drizzle({ client: clientPrimary });
	if (!db) db = drizzle({ client });
	return db;
}

Note: You can run turso db show [database_name] --instance-urls to find your primary database url with the Turso CLI.

Wrapping Up

Using a short-lived cookie to direct reads to the primary database in Turso is a simple and effective way to solve read-after-write consistency issues in Serverless web apps.

So if your app is experiencing data inconsistencies, this approach can help ensure your users always get the most current data.

And if you haven’t tried Turso yet, definitely check it out!