NIMStats automatically benchmarks 20 NVIDIA NIM models every hour using GitHub Actions and publishes the results to a beautiful, interactive dashboard. No servers, no subscriptions — just fork, add your API key, and go.

Get your own benchmarking dashboard running in under 5 minutes.

git clone https://github.com/MauroDruwel/NIMStats.git
cd NIMStats

Visit build.nvidia.com → Create a free account → Copy your API key.

In your forked repo: Settings → Secrets and variables → Actions → New repository secret

Actions → Benchmark NVIDIA NIM Models → Run workflow

That's it — your dashboard auto-refreshes every hour. ✨

┌──────────────────── GitHub Actions (every hour) ──────────────────────┐
│                                                                               │
│   ┌─────────────────────┐        ┌─────────────────────┐                    │
│   │  Job 1 — Group A    │        │  Job 2 — Group B    │  (run in parallel) │
│   │  10 NIM models      │        │  10 NIM models      │                    │
│   └──────────┬──────────┘        └──────────┬──────────┘                    │
│              └──────────────┬───────────────┘                               │
│                    ┌────────▼────────┐                                       │
│                    │  Merge + commit │  → history.db updated in repo         │
│                    └─────────────────┘                                       │
└───────────────────────────────────────────────────────────────────────────── ┘
                                     │
                          ┌──────────▼──────────┐
                          │  Static Dashboard   │  rebuilds on each push
                          │  (Pages / Netlify)  │
                          └─────────────────────┘

Parallel jobs = ~50% faster benchmarks ⚡

PROMPT = "Your custom prompt here"

ALL_MODELS = [
    "your/custom-model",
    # ...
]

- cron: '0 */6 * * *'  # Every 6 hours instead of every hour

# Serve the dashboard
python3 -m http.server 8000
# Open http://localhost:8000

# Run benchmarks manually (requires NIM_API_KEY env var)
export NIM_API_KEY=your_key_here
python3 scripts/test_models.py

history.db is a SQLite database persisted in the repo — the single source of truth. The browser loads it via sql.js (WebAssembly) and queries it entirely client-side. scripts/results.json is a temporary per-job artifact that is never committed.

Schema:

runs          (id, timestamp, prompt, success_count, total_models, fastest_model, fastest_time)
model_results (run_id, model, success, error, response_time, tokens_generated, total_tokens, response)

Benchmark parameters: temperature: 0.7 · top_p: 0.9 · max_tokens: 500 · OpenAI-compatible API

Contributions are what make the open-source community amazing. Any contribution you make is greatly appreciated!

1. Fork the repository
2. Create your feature branch: git checkout -b feat/amazing-feature
3. Commit your changes: git commit -m 'feat: add amazing feature'
4. Push to the branch: git push origin feat/amazing-feature
5. Open a Pull Request

Ideas for contributions:

• 🆕 Add new NIM models to the benchmark list
• 📊 New chart types or dashboard widgets
• 🌐 Internationalization / translations
• 🐛 Bug fixes and performance improvements
• 📖 Improve documentation

Please read through open Issues before starting — someone might already be working on it!

• NVIDIA NIM API Documentation
• NVIDIA Model Catalog
• GitHub Actions Docs
• sql.js — SQLite in the browser

Distributed under the MIT License. See LICENSE for details.