Skip to content

Latest commit

Β 

History

History
350 lines (283 loc) Β· 14 KB

File metadata and controls

350 lines (283 loc) Β· 14 KB

LiveTemplate Architecture

Overview

LiveTemplate is a reactive web framework for Go that uses tree-based DOM diffing to send minimal updates to clients. This document explains the system architecture, design decisions, and operational flow.

New contributor? Start with the Contributor Walkthrough for a hands-on introduction to the 5-phase architecture with code examples and test links.

System Flow

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                     User Action (Browser)                       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                           ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Phase 1: PARSE                                                 β”‚
β”‚  Package: internal/parse/                                       β”‚
β”‚  Input:   Template string                                       β”‚
β”‚  Output:  Parsed AST                                            β”‚
β”‚  Job:     Convert Go template syntax to executable form         β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                           ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Phase 2: BUILD                                                 β”‚
β”‚  Package: internal/build/                                       β”‚
β”‚  Input:   Parsed AST + Data                                     β”‚
β”‚  Output:  Tree structure                                        β”‚
β”‚  Job:     Generate tree separating statics from dynamics        β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                           ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Phase 3: DIFF                                                  β”‚
β”‚  Package: internal/diff/                                        β”‚
β”‚  Input:   Old tree + New tree                                   β”‚
β”‚  Output:  Minimal changes                                       β”‚
β”‚  Job:     Calculate what changed                                β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                           ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Phase 4: RENDER                                                β”‚
β”‚  Package: internal/render/                                      β”‚
β”‚  Input:   Tree or Changes                                       β”‚
β”‚  Output:  HTML (for testing and validation)                     β”‚
β”‚  Job:     HTML rendering and minification                       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                           ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Phase 5: SEND                                                  β”‚
β”‚  Package: internal/send/                                        β”‚
β”‚  Input:   Tree updates + action data                            β”‚
β”‚  Output:  Serialized JSON messages                              β”‚
β”‚  Job:     Message parsing and serialization                     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                           ↓
                    Client applies update

Package Structure

Package Phase Responsibility
internal/parse/ 1: Parse Template parsing into tree structures via custom AST evaluation
internal/build/ 2: Build Tree types, fingerprinting, wrapper injection
internal/diff/ 3: Diff Tree comparison and minimal update generation
internal/render/ 4: Render HTML rendering and minification
internal/send/ 5: Send Action message parsing, update serialization
internal/keys/ Support Sequential key generation for range items
internal/session/ Support WebSocket connection registry with async write pump
internal/observe/ Support Operational metrics and Prometheus export
internal/context/ Support Template execution context
internal/upload/ Support File upload infrastructure
internal/uploadtypes/ Support Upload type definitions
internal/discovery/ Support Template file auto-discovery
internal/compat/ Support Backward compatibility wrappers
internal/util/ Support String utility functions
internal/testutil/ Support Test utilities (Redis helpers)
internal/fuzz/ Support Fuzz testing framework
pubsub/ Top-level Redis pub/sub for distributed broadcasting

For the complete file-by-file map with line counts and dependencies, see CODE_STRUCTURE.md.

Design Decisions

Why Tree-Based Updates?

Problem: Sending full HTML on every update wastes bandwidth

Solution: Separate static HTML (structure) from dynamic data (values)

Result:

  • First render: Send complete tree with statics ["<div>", "</div>"] + dynamics
  • Updates: Send only changed dynamics, client has statics cached
  • Bandwidth reduction: 50-90% for typical templates

Example:

// First render
{
  "s": ["<div>Counter: ", "</div>"],
  "0": "5"
}

// Update (client has statics)
{
  "0": "6"
}

Why HTTP-First (WebSocket Optional)?

Decision: All features work over HTTP, WebSocket only for broadcasts

Rationale:

  • βœ… HTTP works everywhere (no proxy/firewall issues)
  • βœ… Simpler deployment and debugging
  • βœ… Stateless until you need state
  • βœ… WebSocket adds complexity, only needed for server-initiated updates

When to use WebSocket:

  • Real-time collaboration
  • Live notifications
  • Server-initiated broadcasts
  • Multiple simultaneous updates

Why Server-Side State?

Decision: State lives in Go, not split between client/server

Benefits:

  • βœ… Type safety via Go's type system
  • βœ… No synchronization issues
  • βœ… Simpler mental model
  • βœ… Backend logic stays in backend
  • βœ… No need for API layer

Trade-offs:

  • ❌ Not suitable for offline-first apps
  • ❌ Requires server round-trip for interactions
  • βœ… But: Perfect for admin dashboards, internal tools, forms

Why Operational Phase Packages?

Decision: Packages named parse, build, diff, render, send

Benefits:

  • βœ… Self-documenting code structure
  • βœ… Import list shows execution flow
  • βœ… New developers can follow the pipeline
  • βœ… Clear separation of concerns
  • βœ… Easy to find code (operation β†’ package)

Example:

import (
    "github.com/livetemplate/livetemplate/internal/parse"   // Phase 1
    "github.com/livetemplate/livetemplate/internal/build"   // Phase 2
    "github.com/livetemplate/livetemplate/internal/diff"    // Phase 3
    "github.com/livetemplate/livetemplate/internal/render"  // Phase 4
    "github.com/livetemplate/livetemplate/internal/send"    // Phase 5
)

Code Composition Patterns

Orchestrator β†’ Coordinator β†’ Helper Pattern

Large operations are broken into three levels:

1. Orchestrator (25-35 lines)

  • Public API entry point
  • Calls coordinators in sequence
  • Handles errors and logging
  • Example: diff.Compute()

2. Coordinators (20-30 lines)

  • Handle one aspect of the operation
  • Call multiple helpers
  • Example: computeUpdates(), computeInserts()

3. Helpers (<15 lines)

  • Do ONE thing well
  • Pure functions when possible
  • Example: extractItemKey(), haveSameKeys()

Example: Range Diffing

// Orchestrator (25 lines)
func ComputeRangeDiff(old, new *build.Tree) []RangeOperation {
    oldItems, oldKeys := extractRangeItems(old)
    newItems, newKeys := extractRangeItems(new)

    if isPureReorder(oldItems, newItems, oldKeys, newKeys) {
        return []RangeOperation{createReorderOp(newKeys)}
    }

    ops := []RangeOperation{}
    ops = append(ops, computeUpdates(oldItems, newItems, oldKeys, newKeys)...)
    ops = append(ops, computeInserts(oldItems, newItems, oldKeys, newKeys)...)
    ops = append(ops, computeRemoves(oldItems, newItems, oldKeys, newKeys)...)
    return ops
}

// Coordinator (20-30 lines each)
func computeUpdates(oldItems, newItems []interface{}, oldKeys, newKeys []string) []RangeOperation
func computeInserts(oldItems, newItems []interface{}, oldKeys, newKeys []string) []RangeOperation
func computeRemoves(oldItems, newItems []interface{}, oldKeys, newKeys []string) []RangeOperation

// Helpers (<15 lines each)
func extractRangeItems(tree *build.Tree) ([]interface{}, []string)
func extractItemKey(item interface{}) string
func isPureReorder(oldItems, newItems []interface{}, oldKeys, newKeys []string) bool
func haveSameKeys(keys1, keys2 []string) bool

Benefits:

  • Easy to understand (each function fits on screen)
  • Easy to test (each function independently testable)
  • Easy to modify (change one aspect without affecting others)
  • Self-documenting (function names explain intent)

Performance Characteristics

Template Parsing

  • Frequency: Once per template (cached)
  • Complexity: O(n) where n = template size
  • Optimization: Parsed templates are reused

Tree Building

  • Frequency: Every render
  • Complexity: O(n) where n = data size
  • Optimization: Structure fingerprinting for O(1) statics decision

Tree Diffing

  • Frequency: Every update (not first render)
  • Complexity: O(n) where n = tree node count
  • Optimization: Early exit on no changes, range operation detection

Rendering

  • Frequency: Every render/update
  • Complexity: O(n) where n = output size
  • Optimization: Pre-allocated buffers

Wire Format Optimization

First Render

Client receives complete tree with statics:

{
  "s": ["<div class='card'>", "<span>", "</span>", "</div>"],
  "0": "Title",
  "1": "Description"
}

Client caches statics for reuse in subsequent updates.

Subsequent Updates

Server uses fingerprint-based comparison to determine if client needs statics:

// Simple 4-case logic (inspired by Phoenix LiveView)
func ClientNeedsStatics(oldTree, newTree *TreeNode) bool {
    if oldTree == nil { return true }   // First render
    if newTree == nil { return false }  // Removal
    return CalculateStructureFingerprint(oldTree) != CalculateStructureFingerprint(newTree)
}

When structure is unchanged, client receives only changed dynamics:

{
  "1": "Updated description"
}

Client uses cached statics + new dynamics to rebuild DOM.

Bandwidth savings: ~90% for typical updates (statics are the largest part)

Fingerprint-Based Optimization

The fingerprint approach (v0.8.0+) replaces the previous registry-based tracking:

  • O(1) comparison instead of path-based registry lookups
  • Simpler logic: 4 cases vs 49+ state transitions
  • No registry state: Server compares fingerprints directly
  • "When in doubt, send full tree": Safe fallback for edge cases

Security Considerations

HTML Escaping

  • All dynamic content goes through html/template for automatic escaping
  • No direct HTML injection possible
  • User content is always escaped

WebSocket Authentication

  • Configurable Authenticator interface
  • Anonymous mode for development
  • User-based session isolation for production

Session Management

  • Sessions isolated by user ID
  • Session groups for multi-tab support
  • Automatic cleanup on disconnect

Observability

Structured Logging (slog)

All logs are structured JSON:

{
  "time": "2025-10-31T12:34:56Z",
  "level": "INFO",
  "msg": "template_executed",
  "template": "todos",
  "data_type": "*main.TodoState",
  "duration_ms": 5,
  "user_id": "user-123",
  "session_id": "sess-456"
}

Metrics

Emitted periodically via slog:

  • Counters: actions_processed, templates_executed, etc.
  • Gauges: active_connections, active_groups
  • Histograms: p50/p95/p99 latencies

See OBSERVABILITY.md for complete guide.

Future Optimizations

Planned

  • Streaming updates for large datasets
  • Client-side caching improvements
  • Binary diff format (more compact than JSON)
  • Diff batching (combine multiple small updates)

Under Consideration

  • Multi-user broadcast optimization
  • Partial tree invalidation
  • Template compilation cache
  • WebSocket message compression

Related Documentation