Verified structure
Matched facts that hold
31 unique completed recipes; exact GGUF sizes; repeated full-suite distributions; native one-rep speed rows; reproducible fatal-loop cases; memory-fit failures.
Forensic candidate review · final ledger reconciled
The rejection process was too harsh.
Several candidates were dismissed after a single small-slice pass while the incumbent was protected by mixed routes, temperatures, runtimes, and best-of-multiple-run evidence. The session found useful models—but never ran a fully matched final promotion bracket.
Evidence integrity rail
What was actually represented
31
Completed candidate recipes built in-session
+2
Pre-existing tested controls: QP and APEX-I
7
Incomplete or dry-run recipes—not quality rejects
1×
lateKV35 rebuild; duplicate event, not a new recipe
Strongest defensible conclusion
blk44 q4_K is the leading direct-route weight clue. AGENTKV is the strongest alternate tool-state profile. All-K/V q5 and blk42 q4_K were over-pruned. QualityPlus remains the control—not the scientifically established winner.
Confounded evidence
Useful, not weight-only
Port 8083 rows, temperature-1 hard slices, served API medians, MTP runtime variants, and order-sensitive long-run results.
Missing final cell
No matched promotion bracket
No direct temperature-0 full-69 comparison across QualityPlus, blk44, AGENTKV, blk42, and all-K/V q5 under one locked runtime.
02 / REOPEN QUEUE
What deserves more GPU time
Tier A candidates have unresolved evidence strong enough to require a fair bracket. Tier B candidates are only valuable against a specific capability hypothesis.
01
blk44 routed-down q4_K
Leading finalist · +180 MiB
hard20 85/85/85full 89/88/89TG 26.17
The isolated hard slice was 34/40, fell to 26–28/40 inside canonical full order, then returned to 34/40 when ordered first. That is process-state evidence, not a stable weight collapse. No unique failure justified rejection.
Resolution test
Direct 18081/v1 · thinking on · temperature 0 · full 69 · fresh paired QP control. Keep nmax1/p.75 as a separate live-runtime lane.
Direct 18081/v1 · thinking on · temperature 0 · full 69 · fresh paired QP control. Keep nmax1/p.75 as a separate live-runtime lane.
02
AGENTKV
Alternate tool-state profile · +68 MiB
full 91/90Hermes 76/83/81TG 25.89
The best repeated Tool score came through port 8083, which rewrote prompts and thinking behavior. Later 70/70 rows changed route, suite, temperature, seed, and rendering. That blocks promotion—and equally blocks dismissal.
Resolution test
Same direct temperature-0 full-69 cell as QP and blk44. Never use the 8083 proxy as the final control.
Same direct temperature-0 full-69 cell as QP and blk44. Never use the 8083 proxy as the final control.
03
All-K/V q5_K
Unresolved compact lead · +45 MiB
hard20 78 → 70QP control 72TG 25.99
One pass beat the later QP control 31/40 to 29/40; one rebuild repeat scored 28/40. The first artifact was pruned before a repeat, and the final ledger explicitly called that premature.
Resolution test
At least two clean, paired temperature-0 hard-20 repeats. Advance to full 69 only if the signal persists.
At least two clean, paired temperature-0 hard-20 repeats. Advance to full 69 only if the signal persists.
04
blk42 routed-down q4_K
Under-tested stability alternate
hard20 8032/40TG 26.10
It beat QP by three raw points, had no true loop, and was pruned only because it trailed blk44 on one hot-sampler pass. It never received a repeat, full Tool, Hermes, or long-order test.
Resolution test
Repeat in the matched canonical bracket, then stress the long-order dimension where blk44 remains uncertain.
Repeat in the matched canonical bracket, then stress the long-order dimension where blk44 remains uncertain.
Conditional reopenings
These should not enter the first broad bracket. Rebuild or rerun only when the stated capability is the active objective.
lateKV35 q5_K
Old sentinel 88/100 and full 88 conflict with direct 68 and proxy 57. Reopen only as a route-closed K/V diagnostic.
QP late gate/up FP4
Tool 87, Hermes 84, no fatal loop. Run one matched Hermes repeat if stability is the objective.
APEX late gate/up FP4
Tool 88, Hermes 83, no fatal loop. Rebuild only for a specific APEX gate/up hypothesis.
front-q6 / tail-q5
27/40 versus QP 29/40 and uniquely passed TC-60. Worth one repeat only if TC-60 safety leads.
AGENTK-only
Tied QP 29/40, recovered TC-47/61, no loop, but slower and still failed TC-60.
39.49 TG
Mechanism clue—not a model reopen. The late-routed hybrid produced one unusually fast served 4k decode sample, but quality was 55 and size increased by 2.20 GiB. Reproduce the speed spike in an isolated A/B before spending on the overall recipe.
03 / COMPLETE MODEL LEDGER
Every tested identity
Search and filter all 33 tested model identities. Open any row for its complete recorded quality, speed, memory, disposition, and independent reassessment.
- PP
- Prompt-processing tok/s—not perplexity. No successful perplexity result was recorded.
- TG
- Token-generation/decode tok/s. Native and served TG remain separate.
- MiB / BPW
- GGUF weight size—not VRAM. Runtime memory is explicitly labeled where known.
- NR
- Not recorded, not zero. Native speed rows are one repetition only.
| ID | Model / recipe | Family | Size / BPW | Quality headline | Decision |
|---|
04 / BUILD HISTORY
Untested is not rejected
Seven recipes stopped before a completed independent model existed. They belong in the chronology, but none supports a quality conclusion.
| Recipe | Recorded state / size | Why it stopped | Correct interpretation |
|---|---|---|---|
| APEX aligned-sharededge | Manifest staged; no dry-run | Superseded when the objective shifted toward preserving more FPX | Untested |
| APEX priority FPX | 86,962.74 MiB · 3.70 BPW; quant reached ~layer 8 | Changed multiple variables and was replaced by cleaner A/Bs | Untested |
| APEX q6-ROCm-protected | Dry-run ~87,302 MiB | Saved only ~12 MiB versus K-protected | Economics defer |
| APEX q6-ROCm-protected + edge FP4 | Dry-run ~86,357 MiB | Conflated Q6 protection and FP4 edge changes | Untested |
| QP latedown5 q4, layers 40–44 | 84,626.08 MiB · 3.60 BPW · +900 MiB | Pivoted after the layer42 down-only negative | Untested |
| Broad AGENTKV + gate/up FP4 40–44 | Partial quant; projected ~85,144.46 MiB · 3.62 BPW | Narrowed to 40–42; completed child was ~540 MiB smaller | Untested broad form |
| Global Q3_0_ROCMFPX_AGENT | Global dry-run 136,310.25 MiB · 5.81 BPW | Selective overrides unsupported; global result infeasible | Operationally abandoned |
Identity correction: lowercase/uppercase/numeric AGENT override attempts were failed override experiments, not separate models. lateKV35, AGENTKV-lite, and all-K/V rebuilds were repeat events, not new recipes.
05 / SPEED & MTP EVIDENCE
Keep the runtimes separate
Native target-only speed, served MTP throughput, API median turn time, and predicted TG measure different systems. The tables below preserve that separation.
Native target-only, no-MTP
q8 K/V · b8192/u2048 · full Vulkan0 offload · flash attention · gen 128 · one repetition. Small differences are not statistically reliable.
| Model | TG | PP 2k | PP 4k | PP 8k | Interpretation |
|---|---|---|---|---|---|
| QualityPlus | 26.12 | 204.19 | 255.06 | 290.36 | Control |
| blk44 q4_K | 26.17 | 204.36 | 254.92 | 288.32 | Essentially tied with QP |
| APEX edge FP4 | 26.65 | 205.09 | 257.00 | 288.51 | Quality/loop failure dominates |
| APEX K-protected | 27.01 | 204.86 | 256.20 | 284.89 | Only clear TG lead; fatal loop/size reject |
| AGENTKV | 25.89 | 202.12 | 255.18 | 287.21 | No native speed win |
| all-K/V q5 | 25.99 | 202.89 | 252.91 | 284.58 | No native speed win |
| lateKV35 q5 | 26.00 | 198.61 | 249.31 | 280.66 | Slower than QP |
| blk42 q4_K | 26.10 | 202.79 | 253.92 | 280.51 | Served latency clue not confirmed |
| blk44 + lateKV35 | 26.07 | 181.60 | 263.55 | 295.34 | Mixed one-run PG; failed quality |
| blk44 + lateK35-only | 26.22 | 206.13 | 257.19 | 295.23 | Small PG bump; Tool fell to 65 |
| blk43+44 q4_K | 24.45 | 187.16 | 251.56 | 295.03 | TG and short-context PP regressed |
Matched served-MTP comparison: latedown1 versus QP
The cleanest paired served-speed result in the audit. The layer42 rescue was slower at 4k and did not establish a high-context advantage.
| Model | Context | Rep 1 PP / TG | Rep 2 PP / TG | Average PP / TG | Draft accepted / generated |
|---|---|---|---|---|---|
| latedown1 q4_K | 4k | 280.153 / 25.109 | 290.966 / 26.100 | 285.56 / 25.60 | 86 / 96 |
| QualityPlus | 4k | 312.696 / 29.101 | 323.622 / 27.502 | 318.16 / 28.30 | 104 / 112 |
| latedown1 q4_K | 16k | 272.659 / 26.931 | 276.172 / 25.136 | 274.42 / 26.03 | 106 / 127 |
| QualityPlus | 16k | 300.228 / 26.248 | 303.221 / 25.985 | 301.72 / 26.12 | 94 / 99 |
Complete LC-rescue APEX-edge MTP sweep
Original Ciru filenames identify the model unambiguously. The sweep stopped before n2/p.75 at 16k. Best completed high-context TG was 26.895 at n2/p.45.
| MTP | Context / prompt | PP | TG | Accepted / generated | Acceptance | TTFP ms | Total ms |
|---|---|---|---|---|---|---|---|
| n1 p.45 | 4k / 5,142 | 305.011 | 25.574 | 43 / 54 | .7963 | 16,866.8 | 21,872.4 |
| n1 p.45 | 16k / 16,257 | 290.364 | 24.502 | 44 / 60 | .7333 | 56,010.6 | 61,235.4 |
| n1 p.55 | 4k / 5,142 | 303.423 | 25.760 | 43 / 49 | .8776 | 16,957.0 | 21,925.7 |
| n1 p.55 | 16k / 16,257 | 289.684 | 24.485 | 44 / 56 | .7857 | 56,141.3 | 61,371.4 |
| n1 p.60 | 4k / 5,142 | 303.794 | 28.067 | 51 / 54 | .9444 | 16,936.9 | 21,498.1 |
| n1 p.60 | 16k / 16,257 | 290.113 | 23.826 | 40 / 54 | .7407 | 56,061.9 | 61,435.5 |
| n1 p.65 | 4k / 5,142 | 301.400 | 27.783 | 50 / 53 | .9434 | 17,068.9 | 21,676.7 |
| n1 p.65 | 16k / 16,257 | 288.572 | 23.823 | 40 / 54 | .7407 | 56,358.1 | 61,732.0 |
| n1 p.75 | 4k / 5,142 | 300.716 | 26.145 | 43 / 45 | .9556 | 17,108.3 | 22,004.4 |
| n1 p.75 | 16k / 16,257 | 288.455 | 24.134 | 40 / 48 | .8333 | 56,381.2 | 61,685.4 |
| n2 p.45 | 4k / 5,142 | 304.251 | 28.838 | 65 / 73 | .8904 | 16,909.0 | 21,349.0 |
| n2 p.45 | 16k / 16,257 | 290.196 | 26.895 | 66 / 86 | .7674 | 56,043.3 | 60,805.1 |
| n2 p.55 | 4k / 5,142 | 305.523 | 27.183 | 60 / 74 | .8108 | 16,839.4 | 21,549.2 |
| n2 p.55 | 16k / 16,257 | 291.380 | 23.730 | 49 / 73 | .6712 | 55,816.3 | 61,210.9 |
| n2 p.60 | 4k / 5,142 | 305.166 | 26.196 | 52 / 65 | .8000 | 16,858.7 | 21,745.0 |
| n2 p.60 | 16k / 16,257 | 290.056 | 24.177 | 50 / 69 | .7246 | 56,068.8 | 61,366.0 |
| n2 p.65 | 4k / 5,142 | 304.530 | 24.679 | 45 / 60 | .7500 | 16,893.5 | 22,080.5 |
| n2 p.65 | 16k / 16,257 | 290.932 | 23.718 | 47 / 66 | .7121 | 55,899.5 | 61,297.1 |
| n2 p.75 | 4k / 5,142 | 305.331 | 24.727 | 42 / 46 | .9130 | 16,850.7 | 22,027.4 |
Other served/API observations
| Candidate | Observation | Interpretation guardrail |
|---|---|---|
| front-q6/mid-q5/no-tail | 4k PP/TG 338.39 / 25.57, draft 77/83; 16k 317.55 / 23.59, draft 75/99 | TG regression despite good PP |
| late-routed hybrid | One 4k sample 327.12 / 39.49, draft 84/84 | Single sample; quality 55; reproduce first |
| front-q6/tail-q5 | One uncached 4k 333.95 / 25.37, draft 39/47 | One sample; not a high-speed lane |
| AGENTKV + gate/up 40–42 | Eval workload PP 181.78, predicted TG 31.25 | Rough live metric—not controlled speed |
06 / METHODOLOGICAL RIGOR
Why the ranking cannot close yet
The research-rigor review focused on scope, evidence relevance, exploration integrity, and methodological control. Ten defects materially weakened the rejection chain.
01
Unequal burden of proof
QP retained incumbent status despite TC-60/61 failures and Hermes 85/82/79. Candidates were vetoed for the same failures or compared with QP's best result.
02
Temperature changed at the wrong stage
Canonical Tool quality uses temperature 0. Temperature-1/top-p .95 rows are live-sampler compatibility evidence and cannot reject, demote, or prune weights.
03
Port 8083 was not raw
The proxy merged system messages, rewrote aliases, and injected thinking=false even when metadata said thinking was enabled. Old proxy wins and losses are not weight-only.
04
Incomparable suites were pooled
Full-69, hard-20, four-case sentinels, Hermes-20, and ordered-61 percentages are not commensurate. A score of 80 does not mean the same thing across them.
05
Order and process state were causal
blk44 was 34/40 alone, 26–28/40 embedded, and 34/40 when reordered first. Late loops changed identity across runs and failed immediate warm/API reproduction.
06
Runtime was confounded with weights
blk44 received five valid ordered-prefix passes across MTP variants; QP received one. The invalid 401 score of 11 is excluded. Different runtime settings changed behavior materially.
07
Variance was recognized too late
QP Hermes 85/82/79, AGENTKV 76/83/81, edge 86/79, all-K/V 78/70, and APEX control 75/65 show that one pass is not a deletion rule.
08
Loop definitions drifted
Bounded recovery and incomplete chains were initially called loops. The corrected standard requires repeated same or near-same actions without progress.
09
Speed evidence was mixed and weak
Native no-MTP, served MTP, API medians, and predicted TG were discussed together. Native rows used one repetition, so fine-grained ranking is unsupported.
10
Adaptive selection and deletion risk
Dozens of recipes and slices were tried without locked gates. The one-pass guardrail arrived only after all-K/V q5 had already been prematurely removed.
Stress-runtime evidence, correctly scoped
These temperature-1 ordered-prefix rows are live-behavior evidence, not canonical weight scores. They still show that MTP/runtime choice changes the observed failure pattern.
| Profile | Final | Raw | First 20 | Median ms | Interpretation |
|---|---|---|---|---|---|
| QualityPlus · normal MTP | 87 | 106/122 | 29/40 | 5,788.0 | One valid incumbent pass |
| blk44 · nmax2 | 81 | 99/122 | 28/40 | 5,267.8 | TC-42 loop-like failure |
| blk44 · no MTP | 82 | 100/122 | 24/40 | 6,952.2 | Tail loops cleared; quality cost |
| blk44 · nmax1 p.75 | 84 | 103/122 | 30/40 | 5,557.2 | Best candidate runtime lead |
| blk44 · nmax1 p.75 repeat | 83 | 101/122 | 28/40 | 5,785.1 | TC-42/45/54 and 61/68 passed again |
| blk44 · nmax1 p.90 | 80 | 97/122 | 23/40 | 6,467.3 | Loop-like behavior returned |
07 / FAIR RETEST PROTOCOL
One bracket, two lanes
Resolve canonical quality first. Treat the hot live sampler as a separate compatibility lane. Never let one contaminate the other.
Canonical quality lane
Randomized paired blocks, each candidate adjacent to a fresh QP control.
- QualityPlus control
- blk44 q4_K
- AGENTKV
- blk42 q4_K
- all-K/V q5_K
- lateKV35, resources permitting
Gate: repeated hard-20 first; never delete on one pass. Advance stable signals to full-69, then matched Hermes.
Live-serving lane
Run only after canonical ranking is resolved. This is where temperature 1 / top-p .95 belongs.
- QualityPlus live profile
- blk44 normal MTP
- blk44 nmax1 / p.75
- Canonical winner under live sampler
Guardrail: report compatibility and stability separately. Do not use these rows to demote the underlying weights.
Lock before the first run
RouteDirect 18081/v1
TemplateExact chat template; thinking on
SamplerTemperature 0; same remaining values
SeedsRepeated blocks or ≥2 seeds
RuntimeSame MTP, cache, load state
HarnessSame order, tools, limits, timeouts
ScorerSame version and raw denominators
StabilityEqual pass count for QP and finalist
Speed≥3 reps/context + dispersion
MemoryPeak VRAM/GTT at same 64k load
Loop rulePredeclared repeated-action criterion
DeletionRepeat failure or true domination first
08 / PROVENANCE
Where every claim came from
The local audit transcript was reconciled read-only against the current Ciru frontier ledger and original benchmark summaries. Final-ledger corrections govern classification.
Local audit transcript
D:\codexdata\sessionauditstep\codex-session-step37-candidate-audit-019f3803-6f3a-70d2-ae4e-92ae18d097a9.jsonlHigh-value one-based anchors: 861, 891–892, 942–944, 967, 1028, 1093, 1245, 1340, 1368, 1455, 1488, 1515, 1522, 1529, 1545, 1560, 1581.
Authoritative Ciru records inspected read-only
- /home/crown/bench-results/quality/step37-existing-artifact-frontier-20260706.md1,115-line final frontier ledger.
- /home/crown/bench-results/quality/step37-apex-quality-ab-20260706-NOTES.md
- /home/crown/bench-results/llama/step37-apex-non-mtp-control-20260706-NOTES.md
- /home/crown/bench-results/llama/step37-lc-rescue-20260706/mtp-sweep-64k/summary.tsvComplete LC-rescue sweep.
- /home/crown/bench-results/llama/step37-lc-rescue-20260706/latedown1q4-apex-speed-compare-20260706T124716Z/summary.tsvMatched served-speed comparison.
- /home/crown/bench-results/llama/Native one-repetition rows dated 2026-07-06 and 2026-07-07.
Coverage check passed
Final ledger overrides early classifications while preserving earlier results as explicitly confounded evidence.
Final ledger overrides early classifications while preserving earlier results as explicitly confounded evidence.
33 / 33 tested · 31 / 31 built · 7 / 7 incomplete