English | δΈζ
A unified LLM token cost CLI for usage normalization, cost estimation, and multi-provider pricing. π°
LLM providers report token usage differently. OpenAI says prompt_tokens, Anthropic says input_tokens, Vercel AI SDK says inputTokens. Pricing is scattered across provider pages and changes frequently.
TokenCost solves this:
- Normalize any provider's token usage to a unified struct
- Resolve pricing from LiteLLM catalog, OpenRouter API, or built-in static maps
- Calculate USD cost with a single function call
cargo install tokencost
# Look up pricing
tokencost price gpt-4o
tokencost price qwen3-max
tokencost price kimi-k2.5
# Calculate cost
tokencost cost claude-sonnet-4-20250514 --input 10000 --output 2000
# Normalize any provider's format
tokencost normalize '{"prompt_tokens":100,"completion_tokens":50}'
# Aggregate costs from JSONL
tokencost tally calls.jsonluse tokencost::{normalize_token_usage, estimate_usd_cost, pricing_from_usd_per_million};
use serde_json::json;
// Normalize any provider's token usage format
let raw = json!({"prompt_tokens": 100, "completion_tokens": 50});
let usage = normalize_token_usage(&raw).unwrap();
assert_eq!(usage.input_tokens, 100);
// Calculate cost
let pricing = pricing_from_usd_per_million(3.0, 15.0).unwrap();
let cost = estimate_usd_cost(Some(&usage), Some(&pricing)).unwrap();
println!("Total: ${:.6}", cost.total_usd);| Command | Description |
|---|---|
tokencost normalize <json> |
Normalize token usage from any provider format |
tokencost price <model> |
Look up pricing for a model |
tokencost cost <model> -i N -o N |
Calculate cost for given token counts |
tokencost tally <file> |
Aggregate costs from a JSONL file |
tokencost catalog |
Update pricing catalog caches |
All commands support --json for machine-readable output.
# Price lookup with JSON output
$ tokencost --json price deepseek-chat
{
"model": "deepseek-chat",
"input_usd_per_million": 0.27,
"output_usd_per_million": 1.10,
"cache_read_usd_per_million": 0.07
}
# Cost calculation
$ tokencost cost gpt-4o --input 10000 --output 2000
Input: 10000 tokens x $2.50 / 1M tokens = $0.0250
Output: 2000 tokens x $10.00 / 1M tokens = $0.0200
Total: $0.0450
# Batch aggregation
$ tokencost tally calls.jsonl
ββββββββββββββββββββββββββββ¬ββββββββ¬ββββββββ¬βββββββββ¬ββββββββββββ
β Model β Calls β Input β Output β Cost β
ββββββββββββββββββββββββββββͺββββββββͺββββββββͺβββββββββͺββββββββββββ‘
β claude-sonnet-4-20250514 β 42 β 120300β 30100 β $0.812700 β
β gpt-4o β 18 β 54000 β 12600 β $0.261000 β
β TOTAL β 60 β β β $1.073700 β
ββββββββββββββββββββββββββββ΄ββββββββ΄ββββββββ΄βββββββββ΄ββββββββββββ| Provider | Models | Cache Pricing |
|---|---|---|
| Anthropic | claude-sonnet-4, claude-opus-4, claude-haiku-4.5 | read + write |
| OpenAI | gpt-4o, gpt-4.1, o1, o3, o4-mini, etc. | read |
| gemini-2.5-pro, gemini-2.5-flash, gemini-2.0-flash | reasoning | |
| DeepSeek | deepseek-chat, deepseek-reasoner | read |
| Alibaba Qwen | qwen3-max, qwen-plus, qwen-turbo, qwen-flash, etc. | - |
| Zhipu GLM | glm-5, glm-4.7, glm-4.5, flash (free) | - |
| Moonshot Kimi | kimi-k2.5, kimi-k2, kimi-k2-thinking, etc. | read |
| ByteDance Doubao | doubao-2.0-pro, doubao-1.5-pro, doubao-lite, etc. | - |
| MiniMax | minimax-m2.5, minimax-m2, minimax-01 | - |
| StepFun | step-3, step-3.5-flash, step-2-mini | read |
| Baichuan | baichuan4, baichuan3-turbo | - |
| 01.AI | yi-lightning, yi-large | - |
| iFlytek Spark | spark-max, spark-lite (free) | - |
| Source | Coverage | Cache Strategy |
|---|---|---|
| LiteLLM | 1,200+ models across all providers | Disk cache, 7-day TTL, ETag/Last-Modified |
| OpenRouter | 800+ models | In-memory, requires API key |
tokencost normalizes 15+ field name variants across providers:
| Provider | Input | Output | Reasoning | Cache |
|---|---|---|---|---|
| OpenAI | prompt_tokens |
completion_tokens |
- | - |
| Anthropic | input_tokens |
output_tokens |
- | cache_creation_input_tokens, cache_read_input_tokens |
| Vercel AI SDK | inputTokens |
outputTokens |
reasoningTokens |
- |
prompt_tokens_total |
completion_tokens |
- | - |
All normalized to a single TokenUsage struct.
| Feature | Default | Description |
|---|---|---|
cli |
yes | CLI binary (includes catalog) |
catalog |
yes | Network catalog fetching (LiteLLM, OpenRouter) |
| (none) | - | Pure computation library, zero network dependencies |
# Use as a pure library without network/CLI
[dependencies]
tokencost = { version = "0.1", default-features = false }# From crates.io
cargo install tokencost
# From source
git clone https://github.com/justinhuangcode/tokencost
cd tokencost
cargo install --path .MIT