# EXP-43: grades that judge the guess: can a Cadence brain learn text from a teacher's grades alone?

- Date: 2026-10-10 | Cadence version: 0.80.0 (`19bee92`) | Program iter: iter33
- Script: `exp/EXP-43_judged_grades.py` | Evidence: always-learning arms `exp/EXP-43-full.jsonl/.out` + `exp/EXP-43-full-baselines.json`; live arms `exp/EXP-43-live.json/.jsonl/.out`; pilots `exp/EXP-43-pilot*.out`; collapse diagnostics `exp/EXP-43_diag_latch.py` + `exp/EXP-43-diag-{j0,o0,j2}.txt` | Text: Alice ch.1 (train), ch.2 (held out), as EXP-42

## Summary
- Question: EXP-42's levers 1 + 2 together. Does more experience (10 passes, not 3), or a richer grade, let a Cadence brain that never hears the answer reach a model that is told every character? The richer grade: EXP-42's right/wrong grade is noisy (even the best next character is wrong most of the time, so the best guess and a bad guess both mostly hear -1). A teacher who knows the language can grade the guess itself instead of its luck.
- Setup: the teacher reads the book alongside the student; after grading each guess it is told the character (online counts, backing off to shorter contexts it has not seen) and scores a guess by how often that character has followed this context: its best = +1, never seen here = -1, linear between. The score depends only on the text before the guess. Grades on the student's own guess: outcome (+1 right / -1 wrong, EXP-42), judged (the teacher's score), partial (the mean of the two). Cadence arms (modules (32, 16), arousal on, one-step episodes, EXP-42's seeds): outcome-w1, partial-w1, judged-w1 (last character in, bigram teacher), judged-w3 (last 3 characters in, 4-gram teacher). 10 passes over ch.1; greedy held-out accuracy from a saved copy after passes 1, 3, 10; then ch.2 with learning on (prequential). 3 seeds. Baselines: majority; the teachers' own accuracy (teacher-2 = told-the-answer bigram, teacher-4); a tabular epsilon-greedy learner on the same grades (optimistic start). Chain printed per pass for every arm: grade -> mean reward -> agree (the student's guess = the teacher's best) -> accuracy.
- Honest floor, before any number: the teacher was told the characters, so the knowledge comes from supervision; what this measures is how much of a teacher's knowledge passes to a student through grades alone, the quantity the teaching game prices. With one character of context the student's ceiling is its teacher, by construction.
- Prediction (written before the pilot and the full run; a 600-character smoke run of the code was seen first, too short to read):
  - (P1) More passes alone do not get there: outcome-w1 after 10 passes stays more than .03 below teacher-2 on held-out text.
  - (P2) A judged grade does, for one character: judged-w1 comes within .02 of teacher-2.
  - (P3) Partial credit helps: partial-w1 beats outcome-w1 by more than 2 SEM.
  - (P4) The breakaway shot: judged-w3 beats teacher-2, so a brain never told a character predicts held-out text better than a model told every character but seeing only one.
  - (P5) judged-w3 stays more than .05 below its own teacher (teacher-4).
  - (P6) With 3 characters of context the tabular learner beats Cadence on the same judged grades (a table stores the teacher's best for every context it has seen; a (32, 16) brain cannot hold about 2,000 contexts).
  - Check, not a prediction: outcome-w1 passes 1-3 reproduce EXP-42's w1 numbers exactly (same seeds, same grades).
- Pilots (1 seed, 1 pass, full text, seen before the revision below): outcome-w1 pass 1 = .179, exactly EXP-42's seed 0 (the check holds). judged-w1 learned (agree with the teacher .51 in training, held-out greedy .241). partial-w1 and judged-w3 collapsed below always-space (.086 saying "aaa", .069 saying "tta") with the brain aroused on only 1% and 3% of moments: under live, a brain learns only when an outcome surprises it or reward falls below its usual level, and a deterministic grade stops surprising it once its critic expects the bad grade, so it went calm and stopped learning while wrong. The same two arms learning from every outcome (Brain.step, no arousal) reached .239 and .242 after one pass. Always-learning steps cost 9-13 ms against about 4.5, so passes cut to 6.
- Revised design (in the open, after the pilots, before the full run): the grade comparison runs on an always-learning brain (-always: step, no arousal, EXP-39's protocol), 6 passes, greedy probes after passes 1, 3, 6, 3 seeds, arms outcome-w1, partial-w1, judged-w1, outcome-w3 (new control: is it the grade or the protocol that makes 3 characters usable?), judged-w3. The live protocol (EXP-42's) runs separately at 3 passes, 3 seeds, for outcome-w1, partial-w1, judged-w1, judged-w3.
- Revised predictions (P1-P6 now on the always-learning arms at 6 passes, so P1 reads "after 6 passes", wording otherwise unchanged; P7-P8 new):
  - (P7) The grade does the work for longer context: judged-w3 beats outcome-w3 by more than 2 SEM.
  - (P8) The calm trap holds across seeds under live: partial-w1 and judged-w3 are aroused on under 10% of moments after pass 1 and end below always-space (.202) held out; outcome-w1 reproduces EXP-42's three passes exactly.
- Primary metric (declared while the full run was at pass 2, before any held-out result after pass 1): greedy held-out accuracy after the last pass, from a saved copy (no learning, no exploration), the like-for-like comparison with a teacher's best guess. An always-learning brain samples while it reads, so its live prequential number pays for exploration; it is reported alongside. The tabular learner's held-out number is prequential with its epsilon = .1 exploration, as in EXP-42.
- Breakaway = P4 holds.
- Result, always-learning arms (greedy held-out after 6 passes, 3 seeds): outcome-w1 .202 on every seed (always space); outcome-w3 .202 on every seed; partial-w1 .202 on every seed (seed 0 was .239 after pass 1, .227 after 3, .202 after 6: more passes made it worse); judged-w1 .240 and .274, and seed 1 collapsed onto "a" in pass 3 (.275 after pass 1, .070 after pass 3; it then ran over 10 minutes per pass and was stopped after pass 4); judged-w3 .274 / .266 / .222, mean .254 (SEM .016; seed means .240 / .254 / .254 after passes 1 / 3 / 6). Live prequential numbers are lower because they pay for exploration (judged-w3 .251 / .246 / .206).
- Baselines after 6 passes: majority .202; teacher-2 (told the answer, one character) .292; teacher-4 .506; the tabular learner on the same grades: outcome-w1 .248, partial-w1 .266, judged-w1 .267, outcome-w3 .228, judged-w3 .346 (SEM .001).
- Result, live arms (EXP-42's protocol, 3 passes, 3 seeds): outcome-w1 .225, with all three passes and the held-out number identical to EXP-42 on every seed; partial-w1 .165 (SEM .040; .086 / .202 / .206); judged-w1 .192 (SEM .063; .241 / .268 / .067); judged-w3 .208 (SEM .012). Runs that stayed in the birth state (saying "a", action 0, where every brain starts): 0 of 3 under right/wrong grades, 3 of 9 under judged or partial grades. Aroused share in pass 1: 1-9% in the stuck runs and 4-8% in the healthy outcome-w1 runs, so it does not tell them apart. Samples still loop ("t t t", "tt tt tt").
- Verdict: BOUNDARY, 4 of 8 predictions held. P1 held, by collapse rather than a slow climb. P2 failed (the two healthy judged-w1 seeds average .257, one within .02 of its teacher; the third collapsed). P3 failed (partial credit ends where right/wrong ends, always space). P4 failed: no breakaway for Cadence (.254 against .292), though a lookup table on the same judged grades did break away (.346). P5 held. P6 held (table .346, Cadence .254). P7 held (judged-w3 .254 against outcome-w3 .202, 3.2 SEM). P8 failed (partial-w1 stayed calm and below always-space on the mean; judged-w3 did not, and healthy runs are just as calm). The check held exactly.

## Reflection
- What it means: (1) A brain that learns from every right/wrong grade learns the most common answer. Space is right 20% of the time and is the teacher's own best guess at 48% of positions; every other guess is mostly punished, so the policy walks onto space and stays (9 of 9 runs, both context widths, partial credit included). EXP-42's arousal-gated brain did better on the same grades (.225), learning from fewer, surprising outcomes. (2) A grade that judges the guess avoids that collapse and makes longer context usable: with three characters, .254 against .202, Cadence's best text result so far (EXP-42: .225), still below a model told the answer with one character (.292). (3) The grade is no longer the bottleneck. A lookup table on the same judged grades reached .346, past the told-the-answer bigram, so grades alone can carry enough to beat a model told the answer; a (32, 16) Cadence brain at its default operating point did not take that in. On text, the limit is now the brain. (4) Instability: four runs collapsed onto "a" (action 0, where every brain starts), including the best brain of the experiment mid-run (judged-w1 seed 1, .275 to .070). Cadence documents a saturation latch with this shape (docs/reward.md), but at the precision logged its saturation and trace readings did not separate a collapsed seed from healthy ones, so the mechanism is not pinned down.
- For the teaching game: a teacher's judgement of the guess is worth far more to a learner than the world's right/wrong (the table: .346 against .228 with three characters). That is the "how much a grade tells the student" knob, measured on real text. Honest floor: the judgement came from a teacher that was told the answers.
- Process: (1) The pilot did its job: it caught two collapsed arms before the long run, and the design was revised in the open. But the explanation written into that revision (a deterministic grade puts a live brain to sleep) was weaker than stated: healthy runs are as calm, and the stuck runs were stuck from birth. A mechanism read off one pilot seed needs a healthy control before it goes into a write-up. (2) Launching the live arms beside the full run oversubscribed the CPU and slowed both. (3) The collapsed seed was stopped after pass 4 and the main process with it, so the 6-pass baselines were computed separately with the same code (exp/EXP-43-full-baselines.json). (4) P1's "10 passes" became 6 when always-learning proved 2-3x slower.
- Lesson: learning from every right/wrong outcome teaches the most common answer; a grade that judges the guess carries enough for a lookup table to beat a model told the answer. On text, Cadence's limit is now the brain, not the grade.
- Next: (a) the brain: Cadence's operating point for a 28-way readout (actor rate, scale cap, temperature, the documented latch remedies), tested on judged-w3, the only arm with headroom; (b) lever 4, Cadence's own token-window connectome in place of stacked one-hots; (c) for the market: price the judging teacher (worth about .12 accuracy over right/wrong to a table at three characters).