Phone Tool-Eval Scoreboard
QVAC and llama.cpp phone-targeted GGUF runs compared by score, VRAM, latency, total tokens, prompt-processing speed, decode speed, and per-scenario outcomes.
Crown Citadel Performance Lab
Ciru Inference Lab
Strix Halo local inference benchmarks: throughput, memory, serving, tuning, and quality.
Updated
NixOS • Linux 7.0.1
RADV STRIX_HALO
Live quality dashboard
Open quality lab
UltraQuality 7.61 report
FP6 vs Q6 Hermes report
View models
Articles
Articles about running AI locally: hardware, settings, model behavior, tooling, and lessons from the lab.
Articles
Articles about running AI locally: hardware, settings, model behavior, tooling, and lessons from the lab.
UltraQuality 7.61 BPW: What Changed From STRIX QUALITY
The new Qwable 3.6 27B Chadrock ROCmFPX UltraQuality lane improves HermesAgent-20 and KLD while recovering served-MTP draft behavior against the older 7.37 BPW quality recipe.
Local AI Works When the Stack Is Built for It
A tuned Crown Citadel local stack finished a messy invoice sandbox in 5:49, while an Ollama + OpenClaw run stalled at 27:18 and ran out of context.
Your Local Model Settings Matter More Than You Think
How tuned llama.cpp settings changed prompt processing and generation speed on the same Strix Halo machine, with paired before/after bars and percent lifts.
UMA Memory / Speed Frontier
Peak effective UMA memory on x, standalone decode on y, and selected context speed as color.
Prompt / Prefill vs Decode
Context-side prompt processing on x and standalone decode on y, grouped by model family.
Context Curves
Only models with at least four measured context points and one 64k+ result are drawn, so the lines represent real scaling behavior rather than one-off tests.
Benchmark Atlas
Full benchmark matrix. Context columns show prompt-processing or combined prompt+generation throughput, while standalone decode is kept separate in the TG column.
| Profile | Backend | KV / Shape | 4k | 8k | 16k | 32k | 64k | 80k | 100k | 128k | TG | Memory | Measurements |
|---|
Serving Lab
All archived serving rows normalized by model, quant, test type, and tested flag. Main charts promote only capped winners; this lab keeps the full experiment trail.
Settings Comparison
Best matching settings are shown as columns. Metric rows compare every setting against the best value in this slice.
Peer Class Comparison
One best row per comparable model. Change peer class to compare by parameter/ability tier instead of global speed.
| Model / peer | Selected metric | Vs class best | Decode | Prompt | Latency | Settings |
|---|
Detail rows for the current slice
| Run | Test | Flag | Shape | Decode | Prompt | Latency | Memory |
|---|
Quality Suite Leaderboard
Completed coding and repair rows from the quality benchmark store. Suites require real task coverage before they can appear here; failed or incomplete rows stay out of the chart.
| Profile | Test | Score | Evidence | Run |
|---|
HermesAgent-20 Sweep
Official 20-scenario Hermes agent behavior runs. Hy3 API: Novita via OpenRouter (third-party; may not reflect full Hy3 quality). Score is verifier average across memory, workspace, skills, scheduling, delegation, recovery, and approval-boundary scenarios.
Race Gallery
Each card explains what the race tests before you open it. These are focused head-to-head pages for speculative decoding, long-output behavior, task correctness, run health, and memory pressure.
Method Notes
How to read the dashboard without knowing the benchmark harness internals.