You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Define a minimal ABI for communication (initially JSON serialization; later optimize with compact binary formats).
Provide host functions (retricted) for:
Accessing request metadata.
Querying replica health/load.
Return policy/router/middleware decisions.
Core Host Functions
// Host functions exposed to WASM modules (just an example — might change later)pubmod host_functions {// Worker managementpubfnget_worker_count() -> u32;pubfnget_worker_url(index:u32) -> String;pubfnget_worker_load(index:u32) -> u32;pubfnget_worker_health(index:u32) -> bool;// Request informationpubfnget_request_text() -> String;pubfnget_request_headers() -> String;// JSONpubfnget_request_model() -> String;// Metrics and loggingpubfnlog_info(message:String);pubfnlog_error(message:String);pubfnrecord_metric(name:String,value:f64);// External data accesspubfnget_config(key:String) -> String;pubfnhttp_request(url:String,method:String,body:String) -> String;}
3. Lifecycle & Management
Support hot reload of Wasm modules at runtime.
Provide clear error handling: fallback to default Rust logic if a Wasm module fails.
Add Optional sandbox limits (CPU time, memory usage).
Loading and Initialization
sequenceDiagram
participant M as WasmManager
participant R as Runtime
participant W as WASM Module
participant H as Host Functions
M->>R: Load WASM binary
R->>W: Initialize module
W->>H: Register host functions
H->>W: Return function pointers
W->>M: Module ready
M->>M: Register in registry
Loading
Execution Flow
sequenceDiagram
participant R as Router
participant M as WasmManager
participant W as WASM Module
participant H as Host Functions
R->>M: Execute policy/middleware
M->>W: Call WASM function
W->>H: Access host functions
H->>W: Return data
W->>M: Return result
M->>R: Return processed result
Loading
4. Security and Resource Management
Sandboxing: WASM modules run in isolated environments
Resource Limits: Memory and execution time limits
max_memory: MB
max_execution_time: second
Host Function Restrictions: Limited access to system resources
Example Use Cases
Custom Load Balancing Policy
Random policy written in Javascript (just for example)
// A REALLY simple JS random policyexportclassWasmRandomPolicy{constructor(){this.requestCount=0;console.log('WASM Random Policy initialized');}// main entry point of policyselectWorker(workers){this.requestCount++;console.log(`Random policy selecting worker for request #${this.requestCount}`);if(!workers||workers.length===0){console.log('No workers available');return-1;}// Get healthy workersconsthealthyWorkers=workers.map((worker,index)=>({ worker, index })).filter(({ worker })=>worker.health);if(healthyWorkers.length===0){console.log('No healthy workers available');return-1;}// Select a workerconstrandomIndex=Math.floor(Math.random()*healthyWorkers.length);constselectedWorker=healthyWorkers[randomIndex];console.log(`Selected worker index: ${selectedWorker.index} (URL: ${selectedWorker.worker.url})`);returnselectedWorker.index;}getPolicyName(){return'wasm_random_js';}getPolicyVersion(){return'1.0.0';}getStats(){return{requestCount: this.requestCount,policyName: 'wasm_random_js'};}}
Compile to Wasm
# wasm_wrapper.c is a wrapper for calling function from Wasm
emcc wasm_wrapper.c \
-std=c++17 \
-O3 \
-s WASM=1 \
-s EXPORTED_FUNCTIONS="['_malloc', '_free']" \
-s EXPORTED_RUNTIME_METHODS="['ccall', 'cwrap']" \
-s MODULARIZE=1 \
-s EXPORT_NAME="WasmRandomPolicyModule" \
-o wasm_random_policy.js
Load Wasm file with WasmPolicy and WasmPolicyAdapter in sgl-router
usecrate::policies::wasm::{WasmManager,WasmPolicyAdapter};implPolicyRegistry{pubasyncfnregister_wasm_random_policy(&mutself) -> Result<(),WasmError>{letmut wasm_manager = WasmManager::new();// Load Wasm module from local fs
wasm_manager.load_module(WasmModuleConfig{name:"wasm_random".to_string(),path:"/path/to/wasm-random-policy.wasm".to_string(),module_type:WasmModuleType::Policy,config: serde_json::json!({}),}).await?;let wasm_policy = WasmPolicyAdapter::new(wasm_manager,"wasm_random");self.policies.insert("wasm_random".to_string(),Arc::new(wasm_policy));Ok(())}}structWasmPolicyAdapter{wasm_manager:WasmManager,policy_name:String,}implLoadBalancingPolicyforWasmPolicyAdapter{fnselect_worker(&self,workers:&[Arc<dynWorker>],_request_text:Option<&str>) -> Option<usize>{// Worker info in JSON format, just for example nowlet worker_infos:Vec<WorkerInfo> = workers
.iter().map(|worker| WorkerInfo{url: worker.url().to_string(),health: worker.is_healthy(),}).collect();let workers_json = serde_json::to_string(&worker_infos).ok()?;// Executelet selected_index = self.wasm_manager.execute_policy(&self.policy_name,&workers_json,None,)?;if selected_index >= 0 && (selected_index asusize) < workers.len(){Some(selected_index asusize)}else{None}}fnname(&self) -> &'staticstr{"wasm_random"}}
Implemation Plan
Phase 1: Core Wasm Infrastructure
Integrate Wasm runtime (wasmtime)
Implement basic Wasm manager
Add basic security and resource management
Basic metrics
Create host function interface
Phase 2: Middleware (Auth/Billing/RateLimit/PII/Content Mod) Support
Implement Wasm middleware trait and adapter
Integrate with middleware chain
Added adaptation of auth, billing, ratelimit, PII, and content mod
Phase 3: Docs, tests and examples
Example Wasm middleware (auth/billing/ratelimit/PII/content mod)
Checklist
Motivation
Currently, extensibility is limited to static Rust code or config. This makes it harder for production users to:
WebAssembly (Wasm) provides a strong foundation for safe, portable, and dynamic extensibility:
Proposed Design
1. Wasm runtime integration
2. Host <-> Wasm Bridge
Core Host Functions
3. Lifecycle & Management
Loading and Initialization
sequenceDiagram participant M as WasmManager participant R as Runtime participant W as WASM Module participant H as Host Functions M->>R: Load WASM binary R->>W: Initialize module W->>H: Register host functions H->>W: Return function pointers W->>M: Module ready M->>M: Register in registryExecution Flow
sequenceDiagram participant R as Router participant M as WasmManager participant W as WASM Module participant H as Host Functions R->>M: Execute policy/middleware M->>W: Call WASM function W->>H: Access host functions H->>W: Return data W->>M: Return result M->>R: Return processed result4. Security and Resource Management
Example Use Cases
Custom Load Balancing Policy
Implemation Plan
Phase 1: Core Wasm Infrastructure
Phase 2: Middleware (Auth/Billing/RateLimit/PII/Content Mod) Support
Phase 3: Docs, tests and examples
Phase 4: Policy Support
Phase 5: Advanced Features
Related resources
No response