The UltimateQwen Flash 3.8Strix Showdown

Custom quant · custom runtime

Ciru v4.4.1

The old Ciru Flash build: Ciru's own fitted expert quant on Ciru's own HIP runtime, and the baseline the v5 builds here are measured against.

Weights

Routed experts in Q4_1 (G32, about 5.0 bpw). The rest of the model is a protected core of Q5_K, Q8_0, Q5_1 and BF16 tensors.

Runtime

Native Ciru: a llama.cpp/ggml-derived custom HIP runtime for Strix Halo.

• Experts are fitted, not rounded. An activation-order Qronos solve fits gate/up against routing-weighted calibration activations. It then fits down against the already-quantized SwiGLU input so the error from earlier stages is corrected.
• The 51.2B-parameter n-gram embedding table is kept as exact FP8 in a sidecar and paged from NVMe through a 4 GiB cache, so it never occupies GPU memory.
• IU4 WMMA kernels for the Q4_1 experts, MTP depth-3 speculation with a Q8_0 draft, plus sparse-attention, graph and fused-kernel work.

Custom quant · custom runtime

Ciru v5.0

v4.4.1 with a new expert format. Only the routed experts changed. The other 1,079 tensors are byte-identical, and the file is 6.8 GiB smaller.

Weights

C1 expert format at 4.5175 bpw: 4-bit codes, a G16 group index into a 256-entry FP16 scale/offset table, and FP16 per-row scales.

Runtime

Native Ciru with MTP3, packed IU4 dot products for short expert ops and BF16 tiles for long prefill.

• Each projection gets its own learned scale/offset table, set with 8 Lloyd passes and then refined with the same cross-stage Qronos fit as v4.4.1. That covers 24,576 experts; about 2,300 with too little calibration data fall back to MSE.
• The point of C1 is memory. It holds Q4_1-level quality in fewer bits, which frees UMA headroom for long context.
• An optional fused WMMA scorer speeds up the sparse-attention indexer on long single-stream prompts. The n-gram table stays SSD-paged as in v4.4.1.

Ciru conversion · custom runtime

Ciru-Orca

OrcaRouter's uncensored (refusal-removed) Qwen3.8-Flash checkpoint, rebuilt in Ciru's v5 format and served on the Ciru runtime. It is a conversion, not a Ciru fine-tune.

Weights

The same 4.5175-bpw C1 expert format as Ciru v5.0. The non-expert tensors come from the Orca conversion.

Runtime

Native Ciru v5 runtime, using Orca's own Q8 MTP draft at depth 4.

• Orca changes 149 of the 1,658 tensors relative to Qwen, mostly the 48 expert down banks. The gate/up shards were byte-identical to Qwen's, so Ciru v5's fitted gate/up packets were reused after a SHA-256 check.
• The down projections were refitted from Orca's BF16 weights using Orca's own calibration activations, so the quant follows Orca's behavior rather than the base model's.
• Selected dense projections stay in F32 during the 5-token speculative verify batches, to keep accepted drafts consistent.

Community runtime · Unsloth quant

HaloBox HIP

The reference community setup: Unsloth's public dynamic quant on a llama.cpp fork tuned specifically for the Strix Halo GPU (gfx1151).

Weights

Unsloth UD-Q4_K_XL (Dynamic quant). The quant type varies layer by layer, guided by an imatrix calibration set built for agentic coding, chat and multilingual use. Uses Unsloth's separate Q8_0 MTP draft.

Runtime

HaloBox strix-llama.cpp, a llama.cpp fork for RDNA 3.5.

• RDNA 3.5 MMQ/MMVQ tiles and a compact MUL_MAT_ID path for MoE prefill, fused expert routing, and Gated DeltaNet kernels.
• A WMMA flash-attention path, and adaptive MTP draft length (up to 7 tokens) driven by a running acceptance average.
• Direct-I/O model loading, with the large n-gram table kept on the CPU side.

Standalone engine · Unsloth quant

Gufo

The same Unsloth weights as HaloBox on a completely different engine. Comparing the two isolates what the runtime contributes.

Weights

Unsloth UD-Q4_K_XL plus the shared Q8_0 MTP draft, identical files to HaloBox.

Runtime

Gufo: a standalone C++/HIP inference engine, not a llama.cpp fork.

• Its own GGUF reader and dequantization, with kernels written per model instead of generic ones shared across architectures.
• Adaptive MTP speculation (up to 7 draft tokens), chunked prefill and a prefix cache.
• Q4 expert grouping, wave64 gate/up kernels, integer-WMMA Q8 verification and batched DeltaNet recurrence.

Custom quant · HaloBox runtime

AgentionAI AP-Q5_K_XL

AgentionAI's precision-reallocated quant served on the same HaloBox runtime, so the comparison with HaloBox mostly isolates the weights.

Weights

AP-Q5_K_XL (Agention Precision): standard llama.cpp types chosen per tensor group with imatrix. Expert gate/up in Q5_K, about 5.46 bpw overall.

Runtime

HaloBox strix-llama.cpp with the same flags as HaloBox, including Unsloth's Q8_0 MTP draft.

• It spends more bits than the 4-bit stacks to buy accuracy, giving it the highest bits per weight in the field.
• The XL variant differs from AP-Q5_K_M only in the precision of the n-gram table (35.8 vs 26.8 GiB).
• Uses only standard llama.cpp quant types, so any llama.cpp build can load it.

Closed-source quant + engine

Halogen

A fully proprietary stack: Peonist-ai's own weight format and a hand-written HIP engine built for this single model family.

Weights

W4B in Peonist's .hgn format: 4-bit Q4C-P (per-column groups) for trunk and experts, an FP8 paged n-gram table, plus a 2.4 GiB quality overlay.

Runtime

halogen-flash-server 0.14.2, a closed-source container with hand-written gfx1151 HIP kernels. It is not llama.cpp.

• The quality overlay re-quantizes 723 non-expert tensors activation-aware, and keeps o_proj and the MTP head's projections at 8-bit.
• Native MTP plus prompt-lookup self-drafting; greedy output is stated to be byte-identical to non-speculative decoding.
• In the Hermes runs its output was capped at 45,056 tokens to fit Sozo's memory.

Custom quant · matching llama.cpp fork

ROCmFPX2

A custom low-bit format and the llama.cpp fork built to run it. The model loads only on ROCmFPX, not stock llama.cpp.

Weights

HC-Q8 at about 4.11 bpw in one 92 GB GGUF with the MTP head inside. Most of the network is Q4_0_ROCMI4 (4.25 bpw) and the n-gram table is Q3_0_ROCMFPX (3.5 bpw).

Runtime

ROCmFPX, a llama.cpp fork developing AMD-specific weight formats, run with MTP3 and n-gram speculation (16/8/64).

• The 200 hyper-connection up/down matrices are restored from BF16 to Q8_0; that is where the “HC-Q8” name comes from. 98 injection tensors stay BF16.
• A custom copy kernel keeps HIP graph capture on during decode.
• It is the smallest package in the showdown, the result of the 3.5-bit n-gram table and 4.25-bit body.