# EXP-45: the Stone Age chain, grade vs brain (why grading worked for us and not in V5), and search inside an attempt

- Date: 2026-10-13 | Cadence version: 0.83.0 (`23dba71`), both brains on the 0.81 composition (working_memory_scale 12, decay 0.2) | Program iter: none (teaching track; spawned by Will forwarding Stone Age V5)
- Script: `exp/EXP-45_stone_age_2x2.py` | Evidence: `exp/EXP-45-pilot.json/.jsonl`, `exp/EXP-45-full.json/.jsonl/.out`

## Summary
- Question: EXP-39 (ours, 0.80) solved the 4-step Stone Age chain 12/12 by grading every move; Stone Age V5 (ali12mahmood12345-boop/cadence-stone-age, 0.81) reports grade_only at 0.2% and a corrective teacher at 18.8%. Is the difference the grade (V5's is a third as dense, half the size, added to a bigger end reward) or the brain (V5 uses Cadence defaults with arousal; ours is a tuned delayed-credit point)? And is V5's in-sequence success a learned chain, or search inside an attempt (a wrong action costs nothing but a decision, and a no-knowledge never-repeat searcher finishes 30.4%)?
- Setup: V5's COLD chamber re-implemented in our code (theirs read, not run; replica checked: uniform random through it .122 vs exact .121). 2 brains (ours = key-door protocol-3 point; v5 = compose defaults (48, 24) + arousal) x 4 signals (natural, grade-v5, grade-dense, corrective), one continuing life of 120 attempts (V5's budget), V5's event order. Readings on frozen copies: per-state greedy accuracy; greedy full-chain; sampled full-chain over 100 attempts (V5's headline); the same with the carried context erased before every decision (`working_memory.reset(1)`); and the success a memoryless policy with the measured per-state p(right) would have. Seeds 42-53 (V5 used 42-46).
- RUN GATE: (1) prediction below, written before the pilot. (2) Window: pilot of 3 seeds at the exact run conditions (120 attempts) before the full run; if a factor's effect is at floor or ceiling for a reason unrelated to the question, the budget changes before the full run and that change is written here first. (3) Variance: seeds vary; arms differ only in brain or signal. (4) Metric: per-state greedy accuracy is primary for "learned the chain" (search cannot pass it); sampled full-chain is V5's reading and is read against the never-repeat searcher (.304), not against uniform random (.121). (5) Chain per run: train success -> p(right) per state -> greedy accuracy -> sampled / erased / memoryless.
- Prediction (written before the pilot):
  - Check, not a prediction: v5/corrective lands in V5's band (sampled 10-30%, greedy accuracy below 1 in most seeds), so the replica reproduces their regime.
  - (P1) ours/grade-dense gets all 4 states right in >= 10 of 12 seeds (EXP-39 carries to V5's chamber: WAIT added, strict order, 10 decisions).
  - (P2) v5/grade-v5 stays at sampled <= .05 (V5's 0.2% reproduces).
  - (P3) The brain is the bigger factor: ours/grade-v5 mean greedy accuracy >= .75 while v5/grade-dense mean greedy accuracy < .5.
  - (P4) Search: in v5/corrective, sampled exceeds the memoryless prediction by >= .05, and erasing the carried context before each decision closes at least half of that gap.
  - (P5) No v5-brain arm beats the never-repeat searcher (.304) on sampled full-chain success.
- Decision rule (written before the pilot): P3 holds -> V5's grade_only failed on the operating point (actor step 1.0, critic .3, arousal), so the advice to V5 is "tune the brain, keep the grade"; the reverse (v5/grade-dense high, ours/grade-v5 low) -> the diluted grade is the cause; both low off-diagonal -> they interact and both changes are needed. P4 holds -> V5's full-chain number measures in-attempt search through the carried context, and per-state accuracy should be reported beside it.
- Pilot (3 seeds, 42-44, exact run conditions, `exp/EXP-45-pilot.out`), read before the full run: the replica reproduces V5 exactly where both report, v5/corrective sampled .28 / .24 / .20 on seeds 42 / 43 / 44 (V5's published per-seed 28 / 24 / 20%) and seed 42's per-state p(right) .73 / .10 / .11 / .16 (V5's 72.7 / 10.0 / 11.0 / 16.3), v5/grade-v5 .003 (V5 0.2%). The 2x2 contrast is visible and not at a floor or ceiling that hides the question (ours/grade-dense at ceiling is P1's prediction, not a window problem), so the budget stays at 120 attempts; no design change. Because every run is seeded and the code is unchanged, the full run reuses the pilot's three seeds as rows and adds seeds 45-53.
- Result (12 seeds, 120 attempts, `exp/EXP-45-full.out`; acc = per-state greedy accuracy, SEM in brackets; sampled = full-chain over 100 attempts):

  | arm | acc | all 4 states right | greedy solves | sampled | erased | memoryless |
  |---|---|---|---|---|---|---|
  | ours / natural | .58 (.05) | 0/12 | 1/12 | .350 (.030) | .313 | .340 |
  | ours / grade-v5 | .96 (.04) | 11/12 | 11/12 | .912 (.026) | .866 | .862 |
  | ours / grade-dense | 1.00 (0) | 12/12 | 12/12 | 1.000 | 1.000 | 1.000 |
  | ours / corrective | .96 (.03) | 10/12 | 12/12 | .992 (.004) | .908 | .902 |
  | v5 / natural | .33 (.06) | 0/12 | 0/12 | .047 (.023) | .122 | .106 |
  | v5 / grade-v5 | .33 (.04) | 0/12 | 0/12 | .025 (.021) | .006 | .035 |
  | v5 / grade-dense | .42 (.06) | 1/12 | 0/12 | .012 (.012) | .000 | .120 |
  | v5 / corrective | .42 (.04) | 0/12 | 2/12 | .227 (.032) | .098 | .149 |

  Replica check held: v5/corrective on V5's five seeds 42-46 gives .28 / .24 / .20 / .11 / .11, V5's published 28 / 24 / 20 / 11 / 11% exactly. P1 HELD (12/12). P2 HELD (.025). P3 HELD, by a wide margin: ours/grade-v5 .96 against v5/grade-dense .42; V5's own diluted grade solves the chain on our brain (11/12), and grading every move does not rescue V5's brain (0/12 greedy solves), whose greedy answer is the same action in all four states in 6 of 12 dense-graded seeds and 8 of 12 grade-v5 seeds, most often "gather wood" everywhere. P4 HELD as written, with one noisy leg: sampled exceeds memoryless by .077 (SEM .059, about 1.3 SEM), and erasing the carried context drops success .227 to .098 (difference .128, SEM .042, about 3 SEM), past the memoryless .149. P5 HELD: the best v5-brain arm (.227) stays under the never-repeat searcher (.304). Off-prediction: ours/natural reaches .350 sampled with 0/12 brains right in all four states, so even world reward alone passes the searcher bar on full-chain success without learning the chain.
- Verdict: SUPPORTIVE. All five pre-registered predictions held. Decision rule: P3 holds, so V5's grade_only failed on the operating point, not on the grade; P4 holds, so V5's full-chain number is mostly in-attempt search through the carried context and needs per-state accuracy beside it.

## Reflection
- What it means: the disagreement between our EXP-39 and Stone Age V5 was never about grading. On the same chamber, with V5's exact signals, a brain at Cadence's delayed-credit operating point learns the chain from every teacher signal, including V5's diluted grade and V5's corrective labels, while the composed default brain with arousal learns it from none, and mostly collapses onto one action. And V5's headline reading (sampled full-chain success) is the wrong instrument for "learned the chain": in this chamber a policy with no chain knowledge that never repeats a failure gets .304, our world-reward-only brain gets .350 with no brain right in all four states, and V5's best arm gets its .227 mostly from the carried context steering it within an attempt (erase it before each decision: .098). Per-state greedy accuracy separates the arms cleanly; full-chain success does not.
- Process: the replica was the best decision of the run. Matching V5's published per-seed numbers exactly (all five seeds, plus seed 42's fingerprint) is what makes the 2x2 a test of their setup and not of our re-reading of it. Pre-registered, pilot read before the full run, no design change. Two limits: the "ours" brain differs from V5's on many genes at once (modules (32) vs (48, 24), actor step .1 vs 1.0, critic 5 vs .3, lam and gamma, consolidation, trace, arousal off vs on), so this shows the operating point matters but not which gene; and erasing working memory before every decision is not a clean "no memory" control (erased fell below memoryless, so the erase itself may disrupt the stream), so P4's mechanism reading is "the carried context is doing the work" and not a measured search policy. Ran on 0.83 with the 0.81 composition; 0.82's quiet context and 0.83's episodic memory are not tested here.
- Lesson: before reading a teaching result, compute chance for the instrument, not just for the actions. A free-retry chamber makes "random" a floor that a searcher clears 2.5x over. And when two labs disagree, rebuild the other lab's setup until it reproduces their numbers seed for seed, then change one factor.
- Next: (1) dissect the operating point on V5's brain one gene at a time (actor step first: Cadence's own troubleshooting names the composed 1.0; then critic rate, arousal, width) to find which gene sinks it; (2) the same chamber on 0.82's composition and with 0.83's `memory_choice=True` (EXP-46b); (3) update the reply to V5's author with these numbers, Will-gated.