One engine. Three languages.

The reliable and efficient on-device inference core, available as a native Rust crate, a Swift Package, and a Dart Flutter plugin.

SwiftDart
Rust

The native crate. Zero-copy, Tokio async, runs everywhere Rust does — from macOS and Linux servers to embedded targets.

Swift

UniFFI bindings in an SPM package with a signed XCFramework. iOS, macOS, and tvOS. Local test mode built in.

Dart

Flutter-first plugin on pub.dev. On-device inference for iOS, macOS, and visionOS from a single Dart codebase.

Current scope

Chat-focused: ChatEngine, GgufModelConfig, platform-default model selection, and streaming callbacks. More APIs ship with the engine.

crates.iodocs.rsSwift SDKDart SDK
ondeinference on GitHub

Quick Start

Install.

Add the crate via Cargo. One dependency, no build scripts beyond what Cargo handles automatically.

# via CLI
cargo add onde

# or in Cargo.toml
[dependencies]
onde = "0.1"

Usage

Two lines to infer.

Load a model with the platform default config, send a message, get a result. Everything is async and cancellable.

use onde::inference::{ChatEngine, GgufModelConfig};

let engine = ChatEngine::new();
engine
    .load_gguf_model(
        GgufModelConfig::platform_default(),
        Some("You are a helpful assistant.".into()),
        None,
    )
    .await?;

let result = engine.send_message("Hello!").await?;
println!("{}", result.text);
// completed in 85ms — 100% on device

Coverage

Platform Matrix.

macOS and iOS first. tvOS and visionOS are in the pipeline.

PlatformTargetStatus
macOSarm64Ready
iOSarm64 device, arm64 simulatorReady
tvOSarm64Pending
visionOSarm64Pending