Skip to content

storagesdk/storagesdk

storagesdk

npm version CI license

A unified TypeScript SDK for storage with first-class support for snapshotting, forking across Tigris, Amazon S3, Cloudflare R2, GCS, Azure Blob, Vercel Blob and many more.

npm install @storagesdk/core @storagesdk/adapters
import { Storage } from '@storagesdk/core';
import { tigris } from '@storagesdk/adapters/tigris';

const storage = new Storage({
  adapter: tigris({
    bucket: 'agent-runs',
    accessKeyId: process.env.TIGRIS_ACCESS_KEY_ID,
    secretAccessKey: process.env.TIGRIS_SECRET_ACCESS_KEY,
  }),
});

await storage.upload('hello.txt', 'Hello, storage SDK!', {
  contentType: 'text/plain',
});

const text = await storage.download('hello.txt', { as: 'text' });
const url = await storage.url('hello.txt', { expiresIn: 300 });

const snap = await storage.snapshots.create({ name: 'pre-migration' });
await storage.forks.create({ name: 'agent-runs-exp', fromSnapshot: snap.id });
const fork = storage.forks.get('agent-runs-exp');
await fork.upload('hello.txt', 'mutated in fork only');

What you get

  • Snapshots and forks as primitives. Take a snapshot of a bucket, get a read-only handle, fork from it as a writable branch. Native APIs where available (Tigris); sibling buckets/folders otherwise.
  • Typed escape hatch. storage.raw is typed to the underlying SDK (e.g. S3Client on the S3 adapter) for provider-specific operations storagesdk doesn't surface.
  • Agent-ready. @storagesdk/ai wraps every verb (plus the full snapshot and fork roster) as AI tool definitions for the Vercel AI SDK — hand a Storage to your agent runtime and get a ready-to-register tool set back.
  • Runtime adapter selection. @storagesdk/adapters's root export ships ADAPTERS, buildAdapter(name), and getAdapterEnvVars(name) — CLIs and scripts pick any adapter by name, reading config from adapter-native env vars (TIGRIS_*, S3_*, etc.) with backend-native fallbacks (AWS_*, BLOB_READ_WRITE_TOKEN, GOOGLE_CLOUD_PROJECT).
  • CLI. @storagesdk/cli ships the storage and storagesdk binaries. Read any backend (ls, stat, cat, sign) and inspect snapshots/forks (storage snapshots, storage forks) — with --snapshot/--fork to scope reads. storage adapters discovers every adapter with its env-var spec including backend-native fallbacks. Writes and an mcp subcommand follow.
  • ESM-only, Node 20+. Plain tsc build, no bundler.

Adapters

Adapter Subpath Backend
Tigris @storagesdk/adapters/tigris Tigris — snapshots and forks are first-class via Tigris's native APIs.
S3 @storagesdk/adapters/s3 Amazon S3 and any S3-compatible provider.
R2 @storagesdk/adapters/r2 Cloudflare R2.
GCS @storagesdk/adapters/gcs Google Cloud Storage.
Azure Blob @storagesdk/adapters/azure Azure Blob Storage.
Vercel Blob @storagesdk/adapters/vercel Vercel Blob.
MinIO @storagesdk/adapters/minio MinIO.
GitHub @storagesdk/adapters/github GitHub repository — snapshots are tags, forks are branches, native git refs all the way down.
WebDAV @storagesdk/adapters/webdav Any WebDAV server — Nextcloud, ownCloud, Apache mod_dav, nginx-dav, NAS, pCloud, mailbox.org, kDrive. Snapshots/forks via native server-side COPY.
Fly.io @storagesdk/adapters/fly Fly-managed Tigris buckets — branded alias of the Tigris adapter.
Railway @storagesdk/adapters/railway Railway Buckets — branded alias of the Tigris adapter.
Filesystem @storagesdk/adapters/fs Local node:fs/promises. For development and tests.

For the full, up-to-date list see storagesdk.dev/adapters.

API

class Storage<Raw = unknown> {
  constructor(opts: { adapter: Adapter<Raw> });

  readonly raw: Raw;
  readonly snapshots: { create, list, head, delete, get };
  readonly forks:     { create, list, head, delete, get };

  upload(path: string, body: BodyInput, opts?: UploadOptions): Promise<StorageItemMeta>;

  // download — single signature returns full StorageItem; overloads return typed bodies
  download(path: string, opts?: { signal? }):                            Promise<StorageItem>;
  download(path: string, opts: { as: 'stream', signal? }):               Promise<ReadableStream<Uint8Array>>;
  download(path: string, opts: { as: 'text',   signal? }):               Promise<string>;
  download(path: string, opts: { as: 'bytes',  signal? }):               Promise<Uint8Array>;
  download(path: string, opts: { as: 'blob',   signal? }):               Promise<Blob>;
  download(path: string, opts: { as: 'json',   signal? }):               Promise<unknown>;

  head(path: string, opts?: { signal? }):                                Promise<StorageItemMeta>;
  list(opts?: ListOptions):                                              Promise<ListResult>;
  delete(path: string, opts?: { signal? }):                              Promise<void>;
  copy(from: string, to: string, opts?: { signal? }):                    Promise<void>;
  move(from: string, to: string, opts?: { signal? }):                    Promise<void>;
  url(path: string, opts?: UrlOptions):                                  Promise<string>;
  uploadUrl(path: string, opts?: UploadUrlOptions):                      Promise<UploadUrlResult>;
}

snapshots and forks

storage.snapshots.create(opts?: { name?, signal? }):         Promise<SnapshotInfo>;
storage.snapshots.list():                                    Promise<SnapshotInfo[]>;
storage.snapshots.head(id: string, opts?: { signal? }):      Promise<SnapshotInfo>;
storage.snapshots.delete(id: string, opts?: { signal? }):    Promise<void>;
storage.snapshots.get(id: string):                           ReadOnlyStorage; // .download, .head, .list, .url

storage.forks.create(opts: { name, fromSnapshot?, signal? }): Promise<ForkInfo>;
storage.forks.list():                                         Promise<ForkInfo[]>;
storage.forks.head(name: string, opts?: { signal? }):         Promise<ForkInfo>;
storage.forks.delete(name: string, opts?: { signal? }):       Promise<void>;
storage.forks.get(name: string):                              Storage<Raw>;    // full read/write

uploadUrl — PUT vs POST

// PUT: default. Returns a signed URL the client uploads to with PUT.
storage.uploadUrl('photo.jpg', { expiresIn: 300, contentType: 'image/jpeg' });
// → { method: 'PUT', url, headers? }

// POST: triggered by `maxSize` or `minSize`. Returns a presigned POST URL +
// form fields the browser submits as multipart/form-data. Enforces size and
// content-type bounds server-side.
storage.uploadUrl('photo.jpg', { expiresIn: 300, maxSize: 5_000_000, contentType: 'image/jpeg' });
// → { method: 'POST', url, fields }

Errors

Every operation throws StorageError. The code is a typed union:

type StorageErrorCode =
  | 'NotFound'         // missing key, missing snapshot/fork
  | 'NotSupported'     // adapter doesn't implement this op
  | 'Conflict'         // duplicate fork name, etc.
  | 'Unauthorized'     // 401/403 from the backend
  | 'InvalidArgument'  // bad path, sidecar-suffix collision, etc.
  | 'Aborted'          // caller's AbortSignal fired
  | 'Provider';        // unmapped backend error (cause attached)

Common patterns

Snapshots — read frozen state after live writes

await storage.upload('photo.jpg', 'before');
const snap = await storage.snapshots.create({ name: 'baseline' });
await storage.upload('photo.jpg', 'after');

const reader = storage.snapshots.get(snap.id);
await reader.download('photo.jpg', { as: 'text' });   // 'before'
await storage.download('photo.jpg', { as: 'text' });  // 'after'

Forks — branch and mutate

const snap = await storage.snapshots.create();
await storage.forks.create({ name: 'experiment', fromSnapshot: snap.id });

const fork = storage.forks.get('experiment');
await fork.upload('config.json', JSON.stringify({ flag: true }));
// parent unchanged; fork has its own writable view

forks.create also accepts no fromSnapshot — the fork starts at the parent's live state at creation time.

Signed URLs

await storage.url('photo.jpg', { expiresIn: 300 });          // 5-min GET URL
await storage.uploadUrl('new.jpg', { expiresIn: 300 });      // PUT URL + method

Streaming download

const stream = await storage.download('large.mp4', { as: 'stream' });
// Web ReadableStream<Uint8Array>

Byte-range reads

// Fetch a slice instead of the full object.
const item = await storage.download('video.mp4', {
  range: { offset: 0, length: 65_536 },
});
item.size; // 65536 — the slice length, not the full-object size

// Combines with the `as` overloads.
const bytes = await storage.download('big.bin', {
  as: 'bytes',
  range: { offset: 4096, length: 1024 },
});

Maps to each provider's native range API (Range: bytes=N-M for S3-family, download(offset, count) for Azure, createReadStream({ start, end }) for GCS, the Range header on Vercel). range past EOF returns the bytes that exist — matches HTTP Range semantics.

AbortSignal

const ctrl = new AbortController();
setTimeout(() => ctrl.abort(), 5000);

await storage.upload('big.bin', body, { signal: ctrl.signal });
// throws StorageError({ code: 'Aborted' }) if signal fires

Escape hatch

const storage = new Storage({ adapter: tigris({ bucket: 'agent-runs' }) });
//    ↑ Storage typed with the underlying client end-to-end, no cast needed

await storage.raw.someBackendOp({ /* ... */ });

Examples

Runnable examples live under examples/. Each picks the adapter at runtime via EXAMPLE_ADAPTER; out of the box they run against a local filesystem so you can try them without any setup:

pnpm install
pnpm --filter @storagesdk/examples quickstart
pnpm --filter @storagesdk/examples snapshots
pnpm --filter @storagesdk/examples forks
pnpm --filter @storagesdk/examples agent-with-snapshots  # needs ANTHROPIC_API_KEY for the live agent run

Authoring adapters

@storagesdk/adapters is one set of providers; the SDK is designed for third-party adapters too.

npm install @storagesdk/core
import {
  defineAdapter,
  type Adapter,
  StorageError,
} from '@storagesdk/core/adapter';

export function myAdapter(config: MyConfig): Adapter {
  return defineAdapter({
    name: 'my-backend',
    raw: /* your client */,
    async upload(path, body, opts) { /* ... */ },
    async download(path, opts) { /* ... */ },
    async head(path, opts) { /* ... */ },
    async list(opts) { /* ... */ },
    async delete(path, opts) { /* ... */ },
    async copy(from, to, opts) { /* ... */ },
    async move(from, to, opts) { /* ... */ },
    async url(path, opts) { /* ... */ },
    async uploadUrl(path, opts) { /* ... */ },
    snapshots: { /* create, list, head, delete, get */ },
    forks:     { /* create, list, head, delete, get */ },
  });
}

@storagesdk/core/adapter is the adapter-authoring entry. It exposes:

  • defineAdapter — wraps your implementation with path normalization (leading slashes stripped, empty paths throw) and recursive wrapping for snapshots.get / forks.get returns.
  • Adapter, ReadOnlyAdapter, AdapterSnapshots, AdapterForks — the contract types.
  • Manifest helpers (emptyManifest, readManifest, writeManifest, nextSnapshotId, isInternalKey, MANIFEST_PATH) for copy-based adapters that store snapshot/fork lineage as a sibling location.
  • checkSignal, isAbortError, bridgeSignalToController — abort-handling helpers (Web AbortSignal → SDK AbortController bridge with listener cleanup).
  • toWebStream, readStreamToBytes — stream utilities.

Verifying your adapter

Drop in the conformance test suite:

npm install --save-dev vitest @storagesdk/adapters
// my-adapter.test.ts
import { storageAdapterTestSuite } from '@storagesdk/adapters/test-suite';
import { myAdapter } from './my-adapter.js';

storageAdapterTestSuite({
  name: 'my-adapter',
  adapter: () => myAdapter({ /* config */ }),
});

The suite runs the cross-adapter behavioral tests (upload round-trip, NotFound on missing keys, snapshots/forks contract, AbortSignal short-circuit, etc.) against your adapter. Tests it fails are gaps you need to close.

Contributing

See AGENTS.md for development setup, gates (lint / typecheck / build / test), and the design decisions that aren't up for re-litigation.

License

Apache 2.0.

About

A unified TypeScript SDK for storage with first-class support for snapshotting, forking across Tigris, Amazon S3, Cloudflare R2, GCS, Azure Blob, Vercel Blob and many more.

Topics

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Contributors