The new standard of context compression.
Trim long prompts before inference — keep what matters, cut ~65% of wasted tokens.

Quick start · API · Website · Architecture

by Desenyon

Every oversized prompt burns GPU cycles, grid power, and cooling water on tokens that never needed to run. Collective Compress learns which lines to keep for the current question — under a fixed token budget — using a ~5K-parameter CPU policy. No PyTorch. No GPU. Sub-millisecond eviction before your LLM call.

The marketing site and in-browser demo live in web/. Deploy to Vercel with the root vercel.json, or run locally:

cd web && npm install && npm run dev

Open http://localhost:3000 — WebGL shader hero, live compression demo, API dashboard.

export CC_ADMIN_TOKEN=your-secret-here
export CC_KEY_STORE_FILE=.data/api_keys.json
cargo run -p collective-compress-api

curl -X POST http://localhost:8080/v1/compress \
  -H "X-API-Key: cc_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"context":"long text…","query":"What matters?","budget_ratio":0.35}'

cargo build --release
echo "def foo(): return 42" | ./target/release/collective-compress -q "foo function"
./target/release/collective-compress demo   # middle-truncation failure case

use collective_compress::compress_context;

let result = compress_context(
    "long agent context…",
    "What does fetch return when the row is missing?",
    0.35,
    None,
)?;
println!("{:.1}% KV saved", result.kv_savings_pct());

Long context + question
        ↓
  tokenize + 9-dim feature extraction (CPU)
        ↓
  ~5K-param MLP scores each token
        ↓
  eviction policy keeps budget + sinks + recent tail
        ↓
  compressed text → your LLM

Weights ship as assets/checkpoints/model.json. The browser demo loads the same file from web/public/model.json.

No third-party credentials are bundled. Configure your own secrets via environment variables.

cargo test                    # 15 Rust tests
cd web && npm run build       # Next.js production build
docker build -t collective-compress .

Desenyon — Collective Compress

MIT — see LICENSE.