Home/Node Management

Node Management

DockerManager, BinaryManager, container lifecycle, Traefik, auth service, and config TOML

2
node backends
Docker
primary backend
Binary
native backend
Traefik
reverse proxy

Backend Comparison

merobox supports two backends for running merod nodes. The NodeResolver selects the appropriate backend based on workflow configuration.

DockerManager

  • Uses Docker SDK (docker-py) for container lifecycle
  • Creates isolated containers with networking
  • Manages Traefik reverse proxy + auth sidecar stack
  • Automatic port allocation and mapping
  • Volume management for persistent data
  • Image pulling with tag resolution
  • Container health checks via Docker health
  • Clean removal on teardown (containers + volumes)

recommended

BinaryManager

  • Spawns native merod subprocess directly
  • Manages PID files for process tracking
  • Signal-based cleanup (SIGTERM → SIGKILL)
  • Stdout/stderr capture to log files
  • Direct filesystem access (no volumes)
  • No network isolation between nodes
  • Health checks via HTTP polling
  • Lower overhead, useful for development

development

Docker Container Lifecycle

DockerManager manages the full lifecycle of merod containers, from image pull through cleanup.

pull_image tag resolve create_network bridge/overlay gen_config TOML + ports create_container volumes + env start health wait init / bootstrap merod init cleanup: stop → remove container → remove volumes → remove network

Traefik Integration

When running in Docker mode, merobox deploys a Traefik reverse proxy alongside the merod containers. This provides unified HTTP routing and optional TLS termination.

Stack Components

1

Traefik Proxy

Reverse proxy container that routes HTTP/HTTPS traffic to merod nodes based on Host headers and path prefixes.

2

Auth Sidecar

Authentication middleware container that handles JWT validation for incoming requests before they reach merod.

3

merod Containers

The actual Calimero node containers, registered as Traefik backends via Docker labels.

Port Management

DockerManager handles automatic port allocation to prevent conflicts when running multiple nodes.

  • Swarm port: libp2p swarm listener (default range: 2428+)
  • RPC port: JSON-RPC API endpoint (default range: 2528+)
  • Traefik HTTP: External HTTP entry (default: 80)
  • Traefik HTTPS: External HTTPS entry (default: 443)

Each node gets its own RPC and swarm port. Ports are incremented based on node index within the workflow.

Config TOML Generation

Each merod node requires a config.toml file. DockerManager and BinaryManager both generate this config with workflow-specific settings.

# Generated config.toml
[identity]
  path = "/data/identity"

[network]
  swarm_port = 2428
  server_port = 2528
  bootstrap_nodes = ["/ip4/..."]

[store]
  path = "/data/store"

[near]
  rpc_url = "http://sandbox:3030"
  credentials_dir = "/data/near-credentials"

Config Helpers

apply_bootstrap_nodes

Injects bootstrap node multiaddresses into the config. Used to form the initial peer mesh so nodes can discover each other.

apply_e2e_defaults

Sets sensible defaults for E2E testing: faster timeouts, smaller cache sizes, verbose logging, and disabled rate limits.

apply_near_devnet_config_to_file

Configures the NEAR devnet section: sandbox RPC URL, credentials directory, and relayer account settings.

BinaryManager Detail

For development and CI environments without Docker, BinaryManager provides native process management.

Process Lifecycle

1

Locate Binary

Finds merod binary via PATH or explicit binary_path in workflow config.

2

Generate Config

Writes config.toml to a temporary directory with node-specific ports and settings.

3

Spawn Process

Uses subprocess.Popen with stdout/stderr redirected to log files. PID written to .pid file.

4

Health Check

Polls the node’s HTTP health endpoint until it responds 200 or timeout is reached.

Cleanup Strategy

1

SIGTERM

Send SIGTERM to the process. Wait up to 10 seconds for graceful shutdown.

2

SIGKILL

If the process hasn’t exited after SIGTERM timeout, send SIGKILL for forced termination.

3

PID File Cleanup

Remove the .pid file and clean up temporary config and log directories.

NodeResolver Resolution Order

The NodeResolver follows a priority chain to determine how each node declared in the workflow maps to a running instance.

Registered Remote RemoteNodeManager miss URL (explicit) direct endpoint miss Docker DockerManager miss Binary BinaryManager

Each resolver integrates the AuthManager to obtain JWT tokens for authenticated endpoints. If a node declares remote: true, the RemoteNodeManager is consulted first.

Auth Service (Docker Mode)

When --auth-service is enabled, merobox deploys a full authentication stack alongside the merod containers. This creates three interconnected components managed via Docker.

Stack Components

1

Traefik Proxy

proxy container — routes HTTP traffic, applies authentication middleware, handles Host-based routing to individual nodes.

2

Auth Service

auth container — handles authentication and authorization. Validates credentials, issues JWT tokens, manages user sessions.

3

Docker Networks

calimero_web (external-facing) connects Traefik to node containers. calimero_internal (backend) isolates auth service communication.

Architecture

Internet → Traefik (port 80) → Node Containers (calimero_web network)
                             ↓
                       Auth Service (calimero_internal network)

URL Patterns

With Auth Service

  • http://node1.127.0.0.1.nip.io
  • /auth/login — web login UI
  • /admin-dashboard — admin panel

Without Auth Service

  • http://localhost:2528/admin-dashboard
  • http://localhost:2528/admin-api/

Route Protection

Public

  • /admin-dashboard

Accessible without authentication.

Protected

  • /admin-api/
  • /jsonrpc
  • /ws

Require valid JWT token.

Auth Endpoints

  • /auth/login
  • /admin/

Handled by auth service.

CLI Usage

# Start with auth service
merobox run --auth-service

# Multiple nodes with auth
merobox run --count 2 --auth-service

# Custom auth image
merobox run --auth-service --auth-image ghcr.io/calimero-network/mero-auth:latest

# Use cached auth image
merobox run --auth-service --auth-use-cached

# Stop with auth cleanup
merobox stop --auth-service

Workflow Integration

auth_service: true
Auto-Management Details

When --auth-service is enabled, merobox automatically handles:

  • Traefik image pulled and started with correct port bindings
  • Auth service image pulled and started with environment config
  • calimero_web and calimero_internal networks created
  • Node containers configured with Traefik Docker labels for routing
  • CORS headers enabled for cross-origin requests

Frontend Management

Auth Frontend

  • Fresh (default): CALIMERO_AUTH_FRONTEND_FETCH=1
  • Cached: --auth-use-cached or CALIMERO_AUTH_FRONTEND_FETCH=0
  • Custom: --auth-image

WebUI Frontend

  • Fresh (default): CALIMERO_WEBUI_FETCH=1
  • Cached: --webui-use-cached or CALIMERO_WEBUI_FETCH=0

Embedded Auth (Binary Mode)

When using --no-docker with --auth-mode embedded, the merod binary runs a built-in authentication service that provides JWT-based protection without external containers.

How It Works

  • JWT protection on all endpoints: /jsonrpc, /admin-api/, /ws
  • Built-in auth service at /auth/*
  • Username/password provider enabled by default
  • Persistent storage in <node_home>/auth/ (RocksDB)
  • Auto-creates users on first authentication attempt

Auth Endpoints

1

/auth/health

Check auth service status and readiness.

2

/auth/providers

List available authentication providers.

3

/auth/login

Web-based login UI for interactive sessions.

4

/auth/token

Programmatic token generation for API clients.

Getting a JWT Token

# Request a token via the /auth/token endpoint
curl -X POST http://localhost:2528/auth/token \
  -H "Content-Type: application/json" \
  -d '{
    "auth_method": "user_password",
    "public_key": "your-public-key",
    "client_name": "my-client",
    "timestamp": 1234567890,
    "permissions": [],
    "provider_data": {
      "username": "admin",
      "password": "password123"
    }
  }'

Using Tokens

# Authenticate API requests with the JWT token
curl http://localhost:2528/jsonrpc \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN"

CLI Usage

# Single node with embedded auth
merobox run --no-docker --binary-path /path/to/merod --auth-mode embedded

# Multiple nodes with embedded auth
merobox run --no-docker --binary-path /path/to/merod --count 2 --auth-mode embedded

Workflow Integration

auth_mode: embedded
Storage & Notes
  • Binary mode only — not available with Docker backend
  • Auth data persisted at <node_home>/auth/ using RocksDB
  • Users are auto-created on first authentication attempt
  • Each node maintains its own independent auth database

Docker Image Management

DockerManager provides intelligent image handling with automatic registry detection and smart caching to minimize unnecessary pulls.

Smart Pulling

  • Auto-detection of remote registry images (ghcr.io, Docker Hub, etc.)
  • Only pulls if the image is not already available locally
  • Progress display with layer-by-layer download status

Force Pull

  • CLI: --force-pull
  • YAML: force_pull_image: true
  • Always pulls latest from registry, ignoring local cache

Log Level Configuration

Fine-grained logging control using Rust’s RUST_LOG syntax, supporting both global and per-module levels.

Configuration Methods

CLI

merobox run --log-level info

# Module-specific override
merobox run --log-level "info,calimero_context::handlers::execute=debug"

YAML

log_level: "info,module::path=debug"

Available Levels

  • error — critical failures only
  • warn — warnings and errors
  • info — informational messages
  • debug — detailed debug output (default)
  • trace — maximum verbosity

Custom Config Path

Override the generated config.toml with a custom configuration file. Useful for reproducible setups or non-standard node configurations.

Shared Config

nodes:
  config_path: ./custom.toml

Applied to all nodes declared under the nodes key. Skips automatic initialization when provided.

Per-Node Override

nodes:
  - name: node1
    config_path: ./node1-config.toml
  - name: node2
    config_path: ./node2-config.toml

Each node can specify its own config, overriding the shared setting.

Compatibility Notes
  • Works with both Docker and binary mode backends
  • Not supported with count mode (each node needs a unique config)
  • Skips merod init when a custom config path is provided
  • The file must be a valid TOML config that merod can parse