Step 3.7 Flash — Candidate Audit Report

Comprehensive inspection of every quant recipe / model build recorded in the session candidate audit, including metrics (size, BPW, quality scores, speed/TG/PG, MTP sweeps), stated rejection reasons, and an independent reopen recommendation where promising builds were dismissed too aggressively.

Source session: 019f3803-6f3a-70d2-ae4e-92ae18d097a9 Session file: codex-session-step37-candidate-audit-…jsonl Audit period: 2026-07-06 → 2026-07-08 Report generated: 2026-07-09 Base model: Step-3.7-Flash ROCmFPX Q3 QualityPlus family

1. Executive Summary

Your concern is justified. The testing process drifted into “find a reason not to promote” rather than running a fair promotion bracket. Conditions were mixed across temp 0 vs temp 1, proxy :8083 vs direct :18081, MTP variants, and 20-case vs stress-order vs full suite — then artifacts were often pruned after single-pass gates.

Models catalogued
32+

Anchors, probes, overlays, deleted APEX rejects

P0 reopen now
3

blk44 q4_K · AGENTKV · all-K/V q5

P1 reopen
2

lateKV35 · APEX K-protected

Fair rejects
~20

Clear collapses or size-dominated losses

Bottom line: The failure mode is not “no candidates after a day of searching.” It is no fair final bracket, plus premature artifact deletion after mixed-condition single passes. Several real leads remain: blk44 q4_K, AGENTKV, all-K/V q5, and lateKV35.
VRAM note: This audit almost never records per-candidate live VRAM/GTT. Metrics available are primarily disk quant size (MiB / BPW), occasional Vulkan/GTT fit failures on oversized APEX builds, and speed as served MTP tok/s, native non-MTP TG/PG, or median turn ms.

2. Reopen Priority

P0 — reopen now

blk44 ffn_down → q4_K

Best same-route hard-slice lead (85/85/85). Never fairly rejected — only delayed. Full TE 89/88/89, Hermes 84/81, native TG ~tied with QualityPlus.

P0 — reopen now

AGENTKV

Best full Tool Eval (91/90) at only +68 MiB. Demoted for route/temp confounding (8083 / temp 0), not a clean same-test loss. Needs fair direct full suite.

P0 — reopen now

all-K/V q5

Session audit itself called this over-rejected. Beat QP 20-case (78 vs 72), passed TC-60, very fast median (~3153 ms). Artifact deleted prematurely.

P1 — reopen

lateKV35

Full TE 88, sentinel 88/100, only +10 MiB. Later “rebuild weak” used mixed routes; should not have been final-killed.

P1 — reopen

APEX K-protected

Full TE 89, best native TG ~27.01. Later temp=1 collapse needs fair same-route repeat, not discard. Larger (~86G).

P2 — optional

APEX edge FP4 · blk42 q4_K

Edge: best Hermes (86), larger. blk42: 20-case 80 still beats QP 72; rejected only for losing to blk44.

3. Process Failures

3.1 Single-pass kill + prune

3.2 Route / temp confounding

Run type Typical setup Who it favors in notes
Old full TE “wins” often 8083, temp 0 AGENTKV 91/90
Hard 20-case direct 18081, temp 1, thinking blk44 85, QP 72, AGENTKV 70
Stress-order long ordered prefix QP 87 vs blk44 81–84

These were treated as interchangeable “quality.” They are not.

3.3 Asymmetric burden of proof

3.4 Speed under-measurement

3.5 Language collapse: “rejected” vs “not promoted”

blk44 was KEEP / finalist in the ledger, but conversationally treated like “no candidate.” That is how a strong lead disappeared from the mental scoreboard.

Session’s own corrected guardrail (written after several prunes): Do not final-reject or delete any candidate from a single pass if it beats the same-route control, has strong full Tool Eval (88+), strong Hermes full (84+), a unique safety/loop win, or a meaningful speed+quality lead. Those become “repeat required” leads.

4. Master Catalog

Scores use the session’s own labels. Blank cells mean not recorded / not run. Reopen? is the independent recommendation after re-reading reject rationale.

4.A Anchors / baselines

# Model Size (MiB / BPW / Δ) Quality Speed / MTP Status / reason Reopen?
1 QualityPlus (anchor) 83726.08 / 3.57 / baseline Hermes 85/82/79; TE 88/88/90; direct temp1 20-case 72/72; stress 87 (106/122) Served MTP n_max=2 p_min=0.75: TG ~29.97/29.39/28.58/26.26 @ 2k/4k/8k/16k; burst 4k 31.46; native TG ~26.x KEEP Conservative anchor Control only
2 Upstream APEX-I-Compact ~85G TE 88; Hermes 77; fatal HA-05 loop — REJECT Larger, worse Hermes, fatal loop No
3 APEX K-protected 87313.55 / 3.72 / ~86G TE 89; Hermes 81/78; later direct hard-slice 65 + TC-03 loop Best native TG ~27.01 KEEP→demoted Direct collapse needs fair repeat P1 Yes
4 APEX edge FP4 86368.55 / 3.68 / ~85G Hermes 86/79; TE 87/87 LC-rescue sweep n_max=2 p_min=0.45: 4k 28.84, 16k 26.89 KEEP/test Not compact replacement P2 Maybe

4.B Early APEX “deleted obvious rejects”

# Model Size Quality Speed Reject reason Reopen?
5 LC-rescue APEX edge (older) 86248.71 / 3.67 Hermes 75 — Larger, worse Hermes, no TE win No
6 APEX early FP4 87043.55 / 3.71 Hermes ~83/73; HA-17 50 — Larger than edge/lategate, weaker evidence No
7 APEX mid 35–39 q4 90013.55 / 3.83 HA-07 30; full TE 86 non-MTP HA-07 also 30 Size/value reject after non-HA testing No
8 APEX tail 15–44 q4 oversized HA-07 0 / cache0-u512 30 64k Vulkan/GTT fit cliff Too large / memory-unfit No
9 APEX late gate/up FP4 (pre-QP) 86863.55 / 3.70 TE 88; Hermes 83 — Dominated vs QP/edge on size+quality No
10 latedown1Q4 (Apex late-down → q4) 83906.08 / +180 MiB TE 86; Hermes 82 4k TG 25.60 vs QP 28.30; 16k 26.03 vs 26.12 Larger + slower + worse TE/Hermes No (fair)

4.C QualityPlus-family tensor probes

# Model Recipe Δ Size Quality Speed / MTP Stated reject Reopen?
11 late gate/up FP4 40–44 blk40–44 gate/up → q4_0_rocmfp4_fast 85076.08 / +1350 MiB TE 87; Hermes 84; sentinels 50 not a speed win Larger; TE below QP; no TC-60/HA-07 fix No
12 output/token → q6_0_rocmfpx only out/tok promote 84009.30 / +283 MiB TE sentinel 38; HA-17 loops skipped full Loops + bad sentinel No
13 AGENTKV K/V 0–21 q6_K, rest K/V q5_K; Q/O+FFN q5_K 83794.46 / +68 MiB TE 91/90; Hermes 76/83/81; direct temp1 20-case 70/70 median 6102.9 ms; no formal speed sweep KEEP but demote: Hermes worse; TE win route-confounded P0 Yes
14 AGENTKV-lite only early K/V 0–21 q6_K 83771.46 / +45 MiB 20-case 62; Hermes sentinel 50 none before prune Lost AGENTKV tool signal No
15 attn_gate → q6_0_rocmfpx gate-only 83731.85 / +5.77 MiB TE sentinel 62; Hermes sentinel 50 none Cheap negative control No
16 lateKV35 K/V 35–44 → q5_K 83736.08 / +10 MiB sentinel 88/100; full TE 88; rebuild direct 68 / proxy 57 TG 26.00; PG 198.61/249.31/280.66 Reject/prune; audit: “tool lead,” not trash P1 Yes
17 lateKV22 K/V 22–44 → q5_K 83749.08 / +23 MiB sentinel 88; full TE 80 none Full-suite drop vs lateKV35 No
18 all-K/V q5 all attn K/V → q5_K 83771.08 / +45 MiB 20-case 78 (then rebuild 70); passed TC-60 median 3153.4 ms (vs QP 5722.5) Over-rejected; artifact deleted after one pass P0 Yes
19 front-q6 mid-q5 no-tail 0–21 q6, 22–34 q5, tail q4 83784.46 / +58 MiB 20-case 57; TC-60 fail 4k 25.57 / 16k 23.59 decode Quality collapse + slower No
20 late-routed hybrid late 40–44 down q4_K + gate/up FP4 85976.08 / +2.20 GiB 20-case 55 single 4k decode 39.49 (clue only) Huge size, quality collapse No
21 front-q6 tail-q5 0–21 q6, 35–44 q5, mid q4 83781.46 / +55 MiB 20-case 68 single 4k 25.37 Below QP/AGENTKV No
22 AGENTKV + gate/up FP4 40–42 AGENTKV + gate/up 40–42 84604.46 / +878 MiB 20-case 62 prompt 181.78 / predicted 31.25 (rough) Quality regression + size No
23 all-K/V q5 + q6 trans 18–21 allkvq5 + 18–21 q6 ~+50 MiB class 20-case 62 median ~3103 ms Collapsed vs allkvq5 lead No
24 AGENTV V-side only of AGENTKV 83760.27 / +34 MiB 20-case 60 median 6315.2 ms V alone not the signal No
25 AGENTK K-side only of AGENTKV 83760.27 / +34 MiB 20-case 72 (=QP); recovers TC-47/61 median 6275.4 ms Non-candidate as model; K-direction clue Clue only

4.D Step-imatrix routed-down family

# Model Recipe Size Quality Speed / MTP Stated status Reopen?
26 blk44 ffn_down → q4_K Step imatrix peak tensor only 83906.08 / +180 MiB 20-case 85/85/85; TE 89/88/89; Hermes 84/81; stress 81–84 vs QP 87 native TG 26.17; PG 204.36/254.92/288.32 KEEP finalist not full-suite winner P0 Yes
27 blk44 → q4_0_rocmfp4_fast same tensor, FP4 not K 83861.08 / +135 MiB 20-case 70 — FP4 ≠ q4_K here No
28 blk42 q4_K swap Apex-vs-Step disagreement 83906.08 20-case 80 TG 26.10; PG 202.79/253.92/280.51 Weaker than blk44; not faster P2 Maybe
29 blk44 q4_K + lateKV35 both overlays 83916.08 20-case 68 TG 26.07; PG 181.60/263.55/295.34 Worse than either parent No
30 AGENTKV + blk44 q4_K combined frontier 83974.46 / +248 MiB 20-case 70 no speed clue Broke async/state-chain strengths No
31 blk44 + lateK35-only K-only late35 83911.08 20-case 65 TG 26.22; PG 206.13/257.19/295.23 Recovers TC-61 not TC-60 No
32 blk43+blk44 q4_K broaden late-down 84086.08 / +360 MiB 20-case 62 TG 24.45; PG 187.16/251.56/295.03 Damages robustness that made blk44 good No

5. Detailed Model Cards

Filter by reopen priority:

1. QualityPlus (anchor)

KEEP · control Not a reopen target
Size / BPW
83726.08 MiB · 3.57 BPW
Δ vs baseline
baseline
Full Tool Eval
88 / 88 / 90
Hermes
85 / 82 / 79
Direct temp1 20-case
72 / 72
Stress-order
87 (106/122) · 1 pass only
Served MTP TG
29.97 / 29.39 / 28.58 / 26.26 @ 2k–16k
MTP settings
n_max=2 · p_min=0.75 · b8192/u2048 · q8 KV
Burst / native
4k burst 31.46 (n_max=3 p_min=0.60); native TG ~26.x
VRAM
not systematically recorded
StatusSmallest strong overall/stability baseline. Conservative default — not proven overall capability winner under fair brackets.

13. AGENTKV

KEEP contender P0 reopen
Size / BPW / Δ
83794.46 MiB · 3.57 · +68 MiB
Recipe
K/V 0–21 q6_K; rest K/V q5_K; Q/O + dense FFN q5_K
Full Tool Eval
91 / 90
Hermes full
76 / 83 / 81
TE sentinel
75
Direct temp1 20-case
70 / 70
Proxy 8083 20-case
70
Median turn
6102.9 ms (vs QP 5722.5)
Speed sweep
no formal TG/PG sweep recorded
VRAM
not recorded
Stated reason for not promoting Hermes worse than QP; old full TE win considered route-confounded through 8083 / temp 0; direct route repeated only 70. Session later: not rejected — serious contender needing fair full-suite direct repeat.
Why reopen Best repeated full Tool Eval signal at tiny size cost. Demoted for setup differences, not a clean same-test loss. Stable gains on TC-38/47/49/56/61/62 under old full runs. Do not treat as eliminated.

18. all-K/V q5

REJECT → corrected UNRESOLVED P0 reopen
Size / BPW / Δ
83771.08 MiB · 3.57 · +45 MiB
Recipe
all attention K/V → q5_K
20-case (first)
78 (31/40) vs QP 72
20-case (rebuild)
70
TC-60
passed on good pass
Median turn
3153.4 ms (vs QP 5722.5)
Native speed
“did not confirm” high-speed lane
Full TE / Hermes
skipped after single-pass reject
Artifact
deleted prematurely; recipe/logs remain
Stated rejection (original) TC-03/48/61 failures; not enough to promote after one pass — then treated as reject and pruned.
Audit correction Explicitly labeled over-rejected. Beating same-route control + fast median + no true loop is “repeat required,” not delete. Highest process-failure case in the session.

26. Step-imatrix blk44 ffn_down → q4_K

KEEP · same-route finalist P0 reopen · leading candidate
Size / BPW / Δ
83906.08 MiB · 3.57 · +180 MiB
Recipe
only blk.44.ffn_down_exps.weight → q4_K
Direct temp1 20-case
85 / 85 / 85 (3×)
Full Tool Eval
89 / 88 / 89
Hermes
84 / 81
Ordered full-ish
88 (122/138)
Native TG / PG
TG 26.17 · PG 204.36 / 254.92 / 288.32
Stress MTP n_max≈2
81 (99/122)
Stress no-MTP
82 (100/122)
Stress n_max=1
84, then 83 (best runtime)
Stress n_max=1 p90
80
QP stress baseline
87 (106/122) · only 1 pass
Stated reason for not promoting Larger than QP; not faster; does not reliably fix TC-60; full TE lead small; Hermes not clearly better; stress-order trails QP; some loop-like behavior in variants (not consistent).
Correct classification Leading replacement candidate — not rejected. No unique test justifies rejection. Stress variance under MTP settings is a “not yet clean promotion” result, not a kill. Session admitted process was too harsh and too forgiving of QP.

16. lateKV35 (K/V layers 35–44 → q5_K)

REJECT → diagnostic / HOLD P1 reopen
Size / BPW / Δ
83736.08 MiB · 3.57 · +10 MiB
TE sentinel
88 then 100
Full Tool Eval
88
Hermes
sentinel only 50 (incomplete)
Rebuild direct 20-case
68
Rebuild proxy 20-case
57
Native TG / PG
TG 26.00 · PG 198.61 / 249.31 / 280.66
Artifact
pruned then rebuilt for diagnostics
Why reopen Real tool signal at tiny cost. Full TE 88 with TC-60/61 clues. Hermes was only a 4-case sentinel — insufficient to call “obvious reject.” Rebuild weakness used mixed routes; needs fair same-route repeat if K/V lane returns.

3. APEX K-protected control

KEEP/test → demoted P1 reopen
Size / BPW
87313.55 MiB · 3.72 BPW · ~86G
Full Tool Eval
89
Hermes
81 / 78
Later direct hard-slice
65 + real TC-03 reasoning loop
Native TG
best called ~27.01
Why reopen (conditional) Strong TE + best native decode among recorded candidates, but larger and later direct collapse. Worth a fair same-route temp1 bracket if size budget allows — not a compact default.

4. APEX edge FP4

KEEP / test more P2 optional
Size
86368.55 MiB · 3.68 · ~85G
Hermes
86 / 79 (best single Hermes)
Tool Eval
87 / 87
MTP sweep clue
n_max=2 p_min=0.45 → 4k 28.84 / 16k 26.89
NoteHermes lane only; not a compact replacement for QualityPlus.

28. Step-imatrix blk42 q4_K swap

REJECT relative to blk44 P2 maybe
Size
83906.08 MiB
20-case
80 (still > QP 72)
Native TG / PG
TG 26.10 · PG 202.79 / 253.92 / 280.51
Insight Rejected for being weaker than blk44, not for being worse than QP. Still a real hard-slice lift; supports Step imatrix preference for blk44 over Apex-era blk42 priority.

25. AGENTK (K-side isolation)

REJECT as model Clue only
Size / Δ
83760.27 MiB · +34 MiB
20-case
72 (= QP)
Median
6275.4 ms (slower than QP)
ValueK-side is the useful half of AGENTKV; V-side alone (AGENTV) collapses to 60. Recipe clue, not reopen as main model.

Fair rejects (do not reopen)

No reopen

These either collapsed clearly on same-route gates, were size-dominated, or introduced loops/memory cliffs:

  • APEX-I-Compact — HA-05 fatal loop; Hermes 77
  • Early APEX bulk rejects — early FP4, mid35–39 q4, tail15–44 q4 (GTT cliff), LC-rescue edge Hermes 75
  • latedown1Q4 — TE 86, Hermes 82, slower 4k TG 25.60 vs QP 28.30
  • late gate/up FP4 40–44 — +1350 MiB, TE 87
  • output/token q6_0_rocmfpx — TE sentinel 38, HA-17 loops
  • AGENTKV-lite, attn_gate q6 — sentinels ~62 / Hermes 50
  • lateKV22 — full TE 80 after broader K/V
  • front-q6 variants, hybrid, AGENTKV+gateup, allkvq5+trans, AGENTV — clear collapses
  • blk44 FP4, blk44+lateKV*, AGENTKV+blk44, blk43+44 — interaction / broadening negatives

6. blk44 Runtime Matrix (MTP sweeps)

These are runtime controls on the same weights, not new quant recipes. This is where dismissal got especially harsh.

Control Score Detail Valid passes
QualityPlus stress-order baseline 87 106/122 1 only
blk44 normal MTP (n_max≈2) 81 99/122 1
blk44 no-MTP 82 100/122 (invalid 401 run scored 11 — ignore) 1 valid
blk44 MTP n_max=1 84, 83 best blk44 stress runtime 2
blk44 MTP n_max=1 p90 80 — 1
Session conclusion: No unique test justifies rejecting blk44 q4_K. Stress variance under different MTP settings is not a reject — it is “not yet a clean promotion.” Best experimental promotion path: blk44 + MTP n_max=1.

7. Corrected Ranking (Post-Audit)

  1. Replacement lead: Step-imatrix blk44 ffn_down → q4_K (especially MTP n_max=1)
  2. Tool-state lead: AGENTKV (needs fair direct full suite at temp 1)
  3. Unresolved compact clue: all-K/V q5 (rebuild + same-route repeats)
  4. Low-cost diagnostic lead: lateKV35
  5. Speed/TE curiosity: APEX K-protected (if size budget allows)
  6. Conservative default only: QualityPlus

8. Practical Reopen Plan

Do this

  1. Keep/rebuild live: blk44 q4_K — fair final bracket: same route, same temp, full TE + Hermes + stress-order, MTP n_max=1 and n_max=2, vs QP.
  2. Rebuild: AGENTKV — full TE + Hermes on direct 18081 temp1 only (ignore old 8083 as promotion evidence).
  3. Rebuild: all-K/V q5 — 20-case ×3 + full TE + native TG/PG + served MTP (clearest process error).
  4. Optional: lateKV35 — cheap; only if (2)/(3) need K/V localization.

Do not rebuild

  • Hybrid / gate-up stacks
  • output/token q6
  • attn_gate q6
  • blk43+44 broadening
  • AGENTKV+blk44 overlay
  • blk44+lateKV combos

Unless a new case-level hypothesis appears. Interaction space is hostile; next builds should stay single-variable.

9. Additional Insights

ROCmFPX is the body, not the repair

Best new signal is selective escape from FPX (blk44 down → q4_K), not more FPX everywhere. Direct FP4 replacement on the same tensor fell to 20-case 70.

K-side > V-side

AGENTK ≈ QP total (72); AGENTV collapses to 60. Attention key fidelity carries more of the tool-state signal than value alone.

Combining winners usually destroyed them

AGENTKV+blk44, blk44+lateKV, blk43+44 all regressed. Interaction space is hostile; prefer single-variable next steps.

Imatrix source matters

Step imatrix → blk44; Apex-era path → blk42. blk42’s 80 is still a real lift over QP hard-slice — not a waste signal.

MTP is a stability axis

n_max=1 helped blk44 stress; broader MTP did not create a weight win. Treat MTP knobs as runtime A/B, not as quant recipe proof.

Prune policy was too aggressive

Recipes/profiles remain for many deleted shards, but rebuild cost is high. Guardrail written after the damage: don’t final-reject/delete on one pass when same-route control is beaten.

Hard-slice scoreboard (direct temp1 20-case)

Model Score Notes
blk44 q4_K85 / 85 / 85Best same-route hard slice
blk42 q4_K80Still > QP; weaker than blk44
all-K/V q5 (first)78Over-rejected; beat QP
QualityPlus72 / 72Anchor control
AGENTK72= QP total; different case mix
AGENTKV direct70 / 70Route-confounded vs old full TE 91
blk44 FP4 / AGENTKV+blk44 / allkv rebuild70Not promotion material alone
front-q6 tail-q5 / lateKV rebuild / blk44+lateKV65–68Below anchor
AGENTKV-lite / attn_gate / overlays / blk43+4455–62Clear collapses

10. Source & Scope Notes