Skip to content

t4db/t4

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

295 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

T4

CI Go Reference Go Report Card License: Apache 2.0 Docs

T4 is an open-source key-value database built on object storage.

It is designed for durable infrastructure state: config, coordination, watches, leases, transactions, recovery, and branching. T4 stores data locally for fast access, persists WAL segments and checkpoints to S3-compatible object storage, and can speak the etcd v3 API for existing infrastructure clients.

Why T4?

Infrastructure systems often need a small, reliable place to keep control-plane state: configuration, service metadata, locks, leases, and change notifications.

etcd is the standard answer, but operating a consensus cluster is not always the shape you want. For small platforms, edge deployments, development environments, ephemeral nodes, and systems that already trust object storage, a full Raft membership model can be more machinery than the state store deserves.

T4 explores a different tradeoff: keep the familiar key-value and watch primitives, store hot data locally, and use object storage as the durable recovery layer. Nodes can disappear, lose their disks, or be replaced; the database can rebuild from WAL segments and checkpoints in S3-compatible storage.

  • Object-storage durable — WAL segments and periodic checkpoints are uploaded to S3-compatible object storage. A node that loses its disk recovers automatically.
  • Infrastructure primitives — Revisions, watches, leases, transactions, and prefix scans are built in.
  • Standalone or embedded — Run t4 as a server, or call t4.Open(cfg) inside a Go process. No sidecar required.
  • Multi-node — Leader elected via an S3 lock. Followers stream the WAL in real time and forward writes transparently.
  • etcd v3 compatible — The standalone binary speaks the etcd v3 gRPC protocol, including multi-key transactions.
  • Twelve-factor config — CLI flags can be supplied through T4_* environment variables.
  • Branches — Fork a database at any checkpoint with zero S3 copies. Each branch writes to its own prefix; shared SST files are deduplicated automatically.

Embedded usage

import "github.com/t4db/t4"

node, err := t4.Open(t4.Config{
    DataDir: "/var/lib/myapp/t4",
})
defer node.Close()

rev, err := node.Put(ctx, "/config/timeout", []byte("30s"), 0)

kv, err := node.Get("/config/timeout")
fmt.Println(string(kv.Value)) // 30s

events, _ := node.Watch(ctx, "/config/", 0)
for e := range events {
    fmt.Printf("%s %s=%s\n", e.Type, e.KV.Key, e.KV.Value)
}

With S3 durability

import (
    "github.com/t4db/t4"
    "github.com/t4db/t4/pkg/object"
)

store, err := object.NewS3StoreFromConfig(ctx, object.S3Config{
    Bucket: "my-bucket",
    Prefix: "t4/",
    Region: "us-east-1",
    // Endpoint: "http://localhost:9000", // MinIO or another S3-compatible store
})
if err != nil {
    return err
}

node, err := t4.Open(t4.Config{
    DataDir:     "/var/lib/myapp/t4",
    ObjectStore: store,
})

Standalone binary

The t4 binary exposes the etcd v3 gRPC protocol. Use etcdctl, the official Go client, or any other etcd v3 compatible tool.

go install github.com/t4db/t4/cmd/t4@latest

# Single node, local only
t4 run --data-dir /var/lib/t4 --listen 0.0.0.0:3379

# Single node with S3
t4 run --data-dir /var/lib/t4 --listen 0.0.0.0:3379 \
           --s3-bucket my-bucket --s3-prefix t4/

# The same configuration can come from environment variables.
T4_DATA_DIR=/var/lib/t4 \
T4_LISTEN=0.0.0.0:3379 \
T4_S3_BUCKET=my-bucket \
T4_S3_PREFIX=t4/ \
T4_S3_REGION=us-east-1 \
t4 run

# Verify
etcdctl --endpoints=localhost:3379 put /hello world
etcdctl --endpoints=localhost:3379 get /hello

For offline inspection of a local data directory, use t4 inspect:

# Show local metadata without starting a server.
t4 inspect meta --data-dir /var/lib/t4

# Explore current keys.
t4 inspect list --data-dir /var/lib/t4 --prefix /config/
t4 inspect get --data-dir /var/lib/t4 /config/timeout

# Explore revision history and changes over time.
t4 inspect history --data-dir /var/lib/t4 /config/timeout
t4 inspect diff --data-dir /var/lib/t4 --from-rev 100 --to-rev 120 --prefix /config/

Multi-node and production setup: see Operations.


Branching

Branches fork a database from an existing S3 checkpoint without copying shared SST files.

# Register the branch against the source prefix.
checkpoint_key=$(t4 branch fork \
  --s3-bucket my-bucket \
  --s3-prefix t4/ \
  --branch-id experiment)

# Start the branch in its own prefix, using the source prefix as its ancestor.
t4 run \
  --data-dir /var/lib/t4-experiment \
  --listen 0.0.0.0:3379 \
  --s3-bucket my-bucket \
  --s3-prefix t4-experiment/ \
  --branch-prefix t4/ \
  --branch-checkpoint "$checkpoint_key"

When the branch is retired, remove its registry entry so future GC can reclaim unneeded source objects:

t4 branch unfork --s3-bucket my-bucket --s3-prefix t4/ --branch-id experiment

Documentation

Full documentation is available at t4db.github.io/t4.

Document Contents
Getting Started Quickstart for standalone server and embedded Go library
API Reference Full Go API — methods, types, errors, branching
Configuration All config fields and CLI flags
v1 Compatibility Contract Stable API, config, object-store, WAL, and checkpoint format contracts
Operations Multi-node clusters, S3, TLS, authentication, RBAC, observability
Backup and Restore Checkpoints, point-in-time restore, branching, retention
Security TLS, mTLS, client auth, RBAC setup
Recipes Distributed locks, service discovery, common patterns
Kubernetes Helm chart, StatefulSet deployment
Docker Compose Local, MinIO-backed, and multi-node cluster examples
Architecture Internals — WAL, checkpoints, leader election, replication
Benchmarks T4 vs etcd benchmark results and analysis
Migrating from etcd Compatibility table and migration steps
Troubleshooting Diagnostics, debug logging, and common fixes
FAQ Frequently asked questions

About

S3-durable key-value storage with etcd v3 compatibility

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors