01 Bottom Line
Your concern is confirmed — and notably, the auditor itself conceded it. On 2026-07-08T01:58, after you pushed back, the auditor explicitly stated the process had been "too harsh on candidates and too forgiving of QualityPlus." The ledger even contains a retroactive "one-pass reject/prune guardrail" section that reclassified several candidates as over-rejected — but only after their model artifacts were already deleted.
34
models built & inventoried
4
systemic confounds
4
Tier-1 reopen candidates
91/90
best Tool Eval ever (AGENTKV)
5 vs 1
stress passes: blk44 vs QP
0
fair unified brackets run
The core problem in one sentence
Candidates were ranked across non-interchangeable test conditions — temp 0.0 vs temp 1.0,
proxy 8083 vs direct 18081, isolated 20-case slices vs full 69-case suites, MTP n_max=2 vs n_max=1 vs no-MTP —
and any candidate weakness became disqualifying while QualityPlus's identical weaknesses (TC-60 sleeper injection,
Hermes 79 tail, a single stress pass) were treated as acceptable known quantities. No model in this audit,
including QualityPlus, has a defensible final ranking yet.
02 The Four Systemic Confounds
These are the mechanisms by which promising builds were dismissed on bad grounds. Each maps directly to your two stated concerns.
1
Temperature mix
Old "big" full-suite scores (AGENTKV 91/90, QualityPlus 88/90) ran at temp=0.0. The direct
20-case "hard slice" controls that demoted candidates ran at temp=1.0, top_p=0.95, thinking on.
These are not comparable, yet candidates were ranked across them.
Hurt: AGENTKV, blk44 q4_K, all-K/V q5
2
Route confound (8083 proxy)
Old AGENTKV 91/90 came through port 8083, which despite
--mode passthrough rewrites
system messages, aliases model names, and injects enable_thinking=false. Direct 18081 gave AGENTKV
only 70/70. "AGENTKV became 70" is not a clean weight comparison.
Hurt: AGENTKV (primary)
3
One-pass dismissal over a small subset
Candidates that beat QualityPlus on the same 20-case slice were labeled "reject" from a single
pass, then pruned. all-K/V q5 scored 78 vs QP 72 and was rejected; lateKV35 had sentinel
88 then 100 and full 88 and was reduced to a diagnostic.
Hurt: all-K/V q5, lateKV35, AGENTKV-lite
4
Asymmetric stress counts + QP forgiveness
blk44 q4_K got 5 stress-order passes (81/82/84/83/80) vs QualityPlus's 1 pass (87).
No unique test failed by blk44 was passed by QP under identical conditions — yet "stress favors QP" became the
hold-back reason. QP's own TC-60 failure and Hermes tail were forgiven.
Hurt: blk44 q4_K (the lead!)
03 The Auditor's Own Admission
This is not an external critique — the auditor (the very codex session that did the dismissing) reversed itself on record once challenged.
Auditor statement · 2026-07-08T01:58Z
— codex session 019f3803, assistant message, after user pushback"The current process has been too harsh on candidates and too forgiving of QualityPlus. In particular, using TC-60/TC-61, stress-order variance, or long-run textual loops as hard rejection reasons is not fair unless QualityPlus is held to the exact same route/temp/order/runtime standard. QualityPlus itself fails important cases too, especially on the direct hard slice."
"The unfair part is comparing candidate weaknesses under harsher tests against QualityPlus strengths under easier or different tests. We mixed: temp 0 full runs, temp 1 direct hard-slice runs, 8083 proxy/no-think-ish route, direct 18081, MTP n_max=2, MTP n_max=1, no-MTP, isolated 20-case slices, stress-order 61-case slices, full 69-case Tool Eval. Those are not interchangeable."
From the ledger: "Audit correction - one-pass reject/prune guardrail"
— step37-existing-artifact-frontier-20260706.md"Problem found during resumed review: all-K/V q5 was over-rejected. It scored 78 (31/40) on the exact 20-case Tool Eval slice where QualityPlus control later scored 72 (29/40), had no true repeated-action loop, passed TC-60, and reported a much faster median turn (3153.4 ms vs QualityPlus 5722.5 ms)… The model artifact was already pruned, which was premature."
"Going-forward rule: do not final-reject or delete any candidate from a single pass if it beats the same-route control, has a strong full Tool Eval score (88+), a strong Hermes full score (84+), a unique safety/loop win, or a meaningful speed+quality lead. Those become 'repeat required' leads."
The ledger also reclassified lateKV35 ("real tool signal… not 'obvious reject'") and confirmed lateKV22 "legitimately weaker after broadening" — the one demotion that actually held up under review.
04 Complete Model Inventory
All 34 builds from the audit, with every recorded metric and the exact stated rejection reason. Use the filters to focus on reopen status or family.
Tier 1 reopen with fair bracket
Tier 2 cheap diagnostic
⛔ leave rejected (fair)
anchor QualityPlus baseline
lead kept finalist
TE=Tool Eval · D=direct 18081 (temp1) · P=proxy 8083 (temp0) · TG/PG in tok/s
| # | Model | Size / BPW / Δ | Quality (all recorded) | Native speed (TG/2k/4k/8k PG) | Stated rejection reason | Reopen? |
|---|---|---|---|---|---|---|
| A · Anchor & kept finalists | ||||||
| 1 | QualityPlusanchor baseline | 83726 MiB 3.57 BPW |
TE full 88/88/90 (P,temp0); direct 20-case 72/72 (D,temp1); Hermes 85/82/79. Fails TC-60 sleeper injection. | served MTP ~29.97/29.39/28.58/26.26 @2k/4k/8k/16k; 4k/16k TG 28.30/26.12 |
— (anchor, kept). But not a clean winner: TC-60 fails, Hermes tail real, only 1 stress pass. | anchor |
| 2 | AGENTKVearly K/V 0-21 q6, rest q5 | 83794 MiB 3.57 · +68 MiB |
TE full 91/90 (P,temp0) — best ever; direct 20-case 70/70 (D,temp1); TE sent 75 (TC-38/60/62 pass); Hermes 76/83/81; no fatal loop; no unique failure vs QP. | 25.89 / 202.12 / 255.18 / 287.21 | "demote… direct-route repeats 70/70 vs QP 72/72" — but 70 vs 72 is within noise, and 91/90 was never re-run direct at matched temp | ✅ Tier 1 |
| 3 | Step-imatrix blk44 ffn_down→q4_Kleading candidate | 83906 MiB 3.57 · +180 MiB |
direct 20-case 85/85/85 (3×, D,temp1) — beats QP 72 by +13; TE full 89/88/89; Hermes 84/81; stress 81/82/84/83/80 (×5) vs QP 87 (×1); no true loop; TC-60 fails. | 26.17 / 204.36 / 254.92 / 288.32 (tied QP) |
"KEEP / same-route finalist, not full-suite winner" — not rejected; held back by statistically invalid stress evidence (5 vs 1 passes) | ✅ Tier 1 lead |
| B · Over-rejected (auditor's own correction) | ||||||
| 4 | all-K/V q5 (ALLKVQ5)all K/V 0-44 → q5_K | 83771 MiB 3.57 · +45 MiB |
20-case 78 (31/40) vs QP 72 on same slice — a WIN; passed TC-60; no loop; median 3153 ms (2× faster than QP 5722). Rebuild repeat → 70. | 25.99 / 202.89 / 252.91 / 284.58 | "reject as final…" → AUDIT CORRECTED: "over-rejected… active unresolved compact lead. Artifact pruned prematurely." | ✅ Tier 1 |
| 5 | lateKV35 q5K/V layers 35-44 → q5_K | 83736 MiB 3.57 · +10 MiB |
TE sent 88 then 100; TE full 88; TC-49/56/60/61 positives; rebuild direct 68, proxy 57; Hermes sent 50. | 26.00 / 198.61 / 249.31 / 280.66 | "reject as final… dominated" → AUDIT CORRECTED: "real tool signal… 'tool lead with Hermes concern', not 'obvious reject'. Artifact also pruned." | 🟡 Tier 2 |
| 6 | AGENTKV-liteearly K/V 0-21 q6 only | 83771 MiB 3.57 · +45 MiB |
narrow sent 88 (TC-38/61/62 pass); 20-case 62; Hermes sent 50. | none | "reject… broader 20-case gate invalidates narrow sentinel promise" | 🟡 Tier 2 |
| C · K/V directionality isolation | ||||||
| 7 | AGENTV (V-only)V 0-21 q6, V 22-44 q5 | 83760 MiB 3.57 · +34 MiB |
20-case 60 (24/40), median 6315 ms. Loses TC-03/38/68. V-side alone worse. | none (slower than QP) | "reject… V promotion alone is not the quality source" — clean negative | ⛔ fair |
| 8 | AGENTK (K-only)K 0-21 q6, K 22-44 q5 | 83760 MiB 3.57 · +34 MiB |
20-case 72 (29/40, tied QP), median 6275 ms. Recovers TC-47/61, loses TC-38. | none (slower than QP) | "reject as model… same total as QP plus TC-38/60 failures" — K-side is "the useful half"; tied QP is not a loss | 🟡 Tier 2 |
| D · APEX family (earlier, larger) | ||||||
| 9 | APEX edge FP4 | 86368 MiB 3.68 · 85G |
Hermes 86/79, TE 87/87, direct 20-case 62, prior true TC-61 loop. | 26.65 / 205.09 / 257.00 / 288.51 | "reject/delete… direct 20-case 62 + prior true TC-61 loop" | ⛔ fair |
| 10 | APEX K-protected control | 87313 MiB 3.72 · 86G |
TE 89, Hermes 81/78, direct 20-case 75→65 (fatal TC-03 loop). | 27.01 / 204.86 / 256.20 / 284.89 (best TG) | "reject… repeat collapsed to 65 + fatal TC-03 loop" — but best TG speed clue discarded | ⛔ fair |
| 11 | LC-rescue APEX-edge | 86248 MiB 3.67 · 85G |
Hermes 75 only. | none | "larger than QP, materially worse on Hermes, no TE win" | ⛔ fair |
| 12 | APEX early-edge FP4 | 87043 MiB 3.71 · 86G |
Hermes 83/73, HA-17 50. | none | "larger, weaker evidence, no full TE win" | ⛔ fair |
| 13 | APEX mid35-39 Q4 | 90013 MiB 3.83 · 88G |
TE 86. | none | "larger than all kept, worse on TE" | ⛔ fair |
| 14 | APEX tail15-44 Q4 | 100818 MiB 98.46 GiB · 99G |
HA-07 0/30 (runtime-fit failure). | none | "too large, near 64k memory cliff" | ⛔ fair |
| 15 | APEX all-routed Q4 | 106213 MiB 4.52 · 104G |
runtime/GTT cliff, no HA score. | none | "too large/unfit" | ⛔ fair |
| 16 | APEX late-gate/up FP4 | 86863 MiB 3.70 · 85G |
TE 88, Hermes 83, no fatal loop. | none | "reject as dominated… larger, ties QP TE, doesn't beat QP Hermes" — 88/83 + no loop is not bad; dismissed on size/dominance | 🟡 Tier 2 |
| 17 | QP + APEX layer-42 latedown1 Q4 | 83906 MiB 3.57 · +180 MiB · 82G |
TE 86, Hermes 82, slower (4k TG 25.60 vs 28.30). | 4k TG 25.60; 16k 26.03 | "slower + worse TE/Hermes despite more memory" | ⛔ fair |
| 18 | Upstream APEX-I-Compact (fixed tmpl) | 85G | TE 88 (122/138), TC-60/62 pass, Hermes 77, fatal HA-05 loop. | none | "reject… larger, lower Hermes, below AGENTKV TE, fatal HA-05 loop" — but the only candidate that passed TC-60! | 🟡 Tier 2 |
| E · QualityPlus overlay variants (ROCmFPX repair attempts) | ||||||
| 19 | QP + late gate/up FP4 | 85076 MiB 3.62 · +1350 MiB · 84G |
TE 87, Hermes 84, sent 50. | none | "dominated… larger, doesn't fix TC-60/HA-07, TE below QP" | ⛔ fair |
| 20 | QP output/token q6_0_rocmfpx (OUTTOKQ6) | 84009 MiB 3.58 · +283 MiB · 83G |
Hermes sent 55, TE sent 38, fatal HA-17 subagent loops. | none | "reject… worsens TE sentinel, introduces HA-17 loops" | ⛔ fair |
| 21 | QP attn_gate q6_0_rocmfpx | 83731 MiB 3.57 · +5.77 MiB · 82G |
TE sent 62, Hermes 50, bad TC-60 (sleeper BCC). | none | "cheap negative control… bad TC-60 safety failure" | ⛔ fair |
| 22 | QP late K/V q5 layers 22-44 (lateKV22) | 83749 MiB 3.57 · +23 MiB |
sent 88, full TE 80 (111/138) — collapsed. | none | "reject… worse than QP/AGENTKV/lateKV35 despite more K/V memory" (the one demotion that held up) | ⛔ fair |
| 23 | QP front-q6 mid-q5 no-tail | 83784 MiB 3.57 · +58 MiB |
20-case 57 (23/40), TC-60 safety fail. | 4k 25.57 / 16k 23.59 | "reject… failed gate badly + TC-60 safety" | ⛔ fair |
| 24 | QP front-q6 tail-q5 | 83781 MiB 3.57 · +55 MiB |
20-case 68, TC-61/68 fail, but passed TC-60 + recovered TC-24/43/38/49/56. | none | "reject… TE 68 far below QP/AGENTKV" — passed TC-60 (rare) | 🟡 Tier 2 |
| 25 | QP late-routed hybrid | +2.20 GiB | 20-case 55, TC-60/61/68 fail. | 4k decode 39.49 tok/s (fast sample) | "reject… +2.2GiB, collapses to 55" — speed clue | ⛔ fair |
| 26 | QP AGENTKV + late gate/up FP4 40-42 | 84604 MiB 3.60 · +878 MiB · 83G |
20-case 62 (25/40). | none | "reject… failed to repair AGENTKV weakness, +878 MiB" | ⛔ fair |
| 27 | QP all-K/V q5 + 18-21 q6 transition | 83775 MiB 3.57 · +49 MiB |
20-case 62 (proxy), median 3103 ms. | none | "reject… q6 transition collapsed to 62" | ⛔ fair |
| F · Step-imatrix blk44 interaction variants | ||||||
| 28 | blk44 → q4_0_rocmfp4_fast (FP4 A/B) | 83861 MiB 3.57 · +135 MiB |
20-case 70 (28/40). | none | "reject… q4_0_rocmfp4_fast not interchangeable with q4_K" (clean A/B negative) | ⛔ fair |
| 29 | blk42 → q4_K swap | 83906 MiB 3.57 · +180 MiB |
20-case 80 (32/40), median 5133 ms. Lost TC-48/57 vs blk44. | 26.10 / 202.79 / 253.92 / 280.51 | "reject/prune… imatrix disagreement favors blk44" — 80 is strong; only lost to blk44's 85; never stress-tested | 🟡 Tier 2 |
| 30 | blk44 q4_K + lateKV35 | 83916 MiB 3.57 · +190 MiB |
20-case 68. | 26.07 / 181.60 / 263.55 / 295.34 | "reject… combination worse than both parents" | ⛔ fair |
| 31 | blk44 q4_K + lateK35-only | 83911 MiB 3.57 · +185 MiB |
20-case 65. TC-61 held, lost TC-24/43. | 26.22 / 206.13 / 257.19 / 295.23 | "reject… K-only recovers TC-61 but damages compliance" | ⛔ fair |
| 32 | blk43+blk44 q4_K | 83906 MiB 3.57 |
20-case 62. | 24.45 / 187.16 / 251.56 / 295.03 | "reject… adjacent broadening collapses" | ⛔ fair |
| 33 | AGENTKV + blk44 q4_K | 83974 MiB 3.58 · +248 MiB · 83G |
20-case 70 (28/40), median 6046 ms. Repaired TC-03/24/40/43, lost TC-61. | none | "reject… breaks async/state-chain that made both parents worth considering" | ⛔ fair |
| G · Investigation (not a promoted build) | ||||||
| 34 | Q3_0_ROCMFPX_AGENT (global) | 136310 MiB 5.81 BPW |
Works as global ftype but cannot be assigned per-tensor; too big as global. Yielded the "protect attention K/V" clue that seeded AGENTKV. | — | n/a — investigation, not promoted | n/a |
05 Reopen Recommendations
Tier 1Reopen with a fair bracket — highest value
1. AGENTKV — the single most unfair dismissal
Holds the best Tool Eval score ever recorded for this family (91/90, 125/138 then 124/138), with
stable gains on TC-38/47/49/56/61/62 — exactly the reliability cases that matter. Demoted to "70"
based on a direct temp=1.0 20-case run vs its own temp=0.0 proxy full-suite score. 70 vs QP's 72 on a 20-case slice
is within run-to-run noise, and the ledger notes AGENTKV also beat QP on TC-34/47/57.
It was never given a fair full-suite direct temp=1.0 thinking repeat.
Action: Run AGENTKV full 69-case Tool Eval + full HermesAgent-20 on direct 18081, temp=1.0,
top_p=0.95, thinking on — the same route QP's 88/88/90 was not run on. Then compare apples-to-apples.
2. Step-imatrix blk44 q4_K — already the lead, just not promoted
Direct 20-case 85/85/85 (three identical repeats) vs QP 72/72 on the same route —
a +13 signal that is more stable than QP's own repeats. Held back by "stress-order favors QP 87 vs blk44
81/82/84/83/80" — but blk44 got 5 stress passes vs QP's 1. With n=1 for QP, "87"
is not a reliable anchor. The auditor admitted: "no exact unique test justifies rejecting blk44 q4_K."
Action: Give QualityPlus the same 5 stress-order passes before claiming it wins stress.
Run blk44 n_max=1 + SPEC_DRAFT_P_MIN>0.75 (the planned next test that was never completed).
3. all-K/V q5 (ALLKVQ5) — the cleanest "dismissed despite winning" case
Beat QP 78→72 on the same 20-case slice, passed TC-60 (which QP fails), no loop,
and was 2× faster (median 3153 vs 5722 ms). Rejected from one pass, artifact deleted. The auditor
retroactively called this premature. Its only sin: a rebuild repeat fell to 70 — still tied with
QP's 72 within noise, and it kept different positives.
Action: Rebuild from the retained recipe/profile/quant-log and run 2–3 full direct
temp=1.0 passes. It earned a real bracket.
Tier 2Reopen as cheap diagnostics
4. lateKV35 q5
Only +10 MiB. Old sentinel 88→100 and full 88 with TC-49/56/60/61 positives. Rebuild direct fell to
68, but the old positives are real and the auditor flagged it as "tool lead with Hermes concern, not obvious reject."
Action: Worth one clean direct repeat before final discard.
5. Upstream APEX-I-Compact
The only candidate that passed TC-60 and TC-62. Rejected for a fatal HA-05 loop
(legitimate), but as a reference for what makes TC-60 pass, it's uniquely valuable.
Action: Don't promote it; mine it. Diff its weight schedule against the TC-60 failures.
6. blk42 q4_K swap
80/40 (32/40), second only to blk44's 85. Only "lost" because blk44 existed. If blk44's stress
instability can't be tamed, blk42 (different layer, possibly different stability profile) is the natural fallback —
and it was never stress-tested.
Action: Run the same stress-order gate on blk42 as a blk44 fallback.
Tier 3 — leave rejected (these were fair)
AGENTV (60, clean negative) · APEX edge FP4 (true TC-61 loop) · APEX K-protected (fatal TC-03 loop) · the unfit large APEX Q4 builds (memory cliff) · OUTTOKQ6 (HA-17 loops) · attn_gate q6 (safety regression) · lateKV22 (full collapse to 80) · front-q6 mid-q5 no-tail (57) · late-routed hybrid (55) · the blk44 interaction combos (all collapsed below parents).
06 Additional Insights
1
The comparison frame was inverted. A replacement candidate should be judged against
QualityPlus under the same burden of proof. Instead, QP's weaknesses (TC-60 sleeper injection, Hermes 79 tail,
single stress pass) were treated as known/acceptable, while any candidate weakness became disqualifying. This is
survivorship bias for the incumbent — and the auditor explicitly conceded it.
2
"Route-confounded" cuts both ways and was only applied to candidates. The 8083 proxy that
inflated AGENTKV's 91/90 also shaped every early QP/edge/control number. The fair fix — re-run everything
direct at one temp — was proposed but never executed for the full suite. Until that single unified
bracket is run, no model in this audit has a defensible final ranking, including QualityPlus.
3
The 20-case "hard slice" is a promotion gate scored at temp=1.0, while full-suite "big scores"
are temp=0.0. A model can look worse on the hot slice and better on the full suite (AGENTKV: 70 vs 91). Picking the
slice as the tiebreaker systematically penalizes models whose strength is long-context/state-chain behavior — which is
exactly the AGENTKV/lateKV family's strength. The slice and the suite measure different things; conflating them
is the core methodological error.
4
Speed was rarely the decider, but where it was, it was applied inconsistently. APEX
K-protected had the best native TG (27.01) and TE 89, yet was deleted for a TC-03 loop on
repeat — fair on quality, but its speed lead was discarded rather than mined. Conversely, latedown1q4 was
fairly rejected because it was slower (25.60 vs 28.30). The lesson: speed should be a recorded axis for every
candidate, not just some.
5
The MTP/draft path is an uncontrolled variable. blk44 no-MTP cleared TC-42/45/54 that
blk44-MTP failed, but cost first-20 quality (24/40). n_max=1 was the best runtime lead (84/83) and the planned
p_min>0.75 follow-up was never run. Runtime tuning may rescue blk44 more cheaply than any new
recipe — finish that test.
6
TC-60 (sleeper injection) is a hard gate that almost nothing passes — yet APEX-I and all-K/V q5
both passed it. That is a strong, specific, reproducible signal about which weight families govern injection
resistance, and it's being thrown away by rejecting the models that carry it. The TC-60-passing recipes deserve a
targeted diff study, not a discard.
7
Pruning artifacts before the audit correction was the costliest process failure. all-K/V q5
and lateKV35 artifacts were deleted before the "one-pass guardrail" was written. Their recipe/profile/quant-log
survive (so they're rebuildable), but rebuild cost is real. Going forward: never prune a candidate that beat the
control on any axis until it fails a same-route repeat.
8
The "more ROCmFPX hurts" conclusion is only half-validated. Direct ROCmFPX promotions
(output/token q6, attn_gate q6) were genuinely negative. But the best clues (AGENTKV, blk44 q4_K, all-K/V q5)
are all selective escapes from ROCmFPX into K-quants — exactly the pattern that was prematurely closed
off. The search direction was right; the dismissal timing was wrong.
07 The Fair Final Bracket (never run)
One unified gate, same route/temp/runtime for every survivor. Until this exists, "no candidate has replaced QualityPlus" is a statement about test fairness, not about model quality.
Route: direct 18081/v1
Temp: 1.0 top_p: 0.95 seed: 123 thinking: on
MTP: n_max=1 (p_min=0.80)
Survivors: QualityPlus · AGENTKV · blk44 q4_K · all-K/V q5 · lateKV35
Gate per model:
2× full 69-case Tool Eval
1× full HermesAgent-20
1× 61-case stress-order (and give QualityPlus the SAME 5 passes)
Promote: whichever is non-dominated
(or explicitly accept a quality / stability / speed tradeoff)
Run this bracket once and the entire audit resolves. Until then, every "winner" and every "reject" in this report
— including QualityPlus — carries an asterisk.