Overview · Pre-registration
Four go/no-go tests, written down before the run
Overview
We have four research ideas that would each cost weeks and rented GPUs to pursue properly. Before spending that, each gets a cheap test ("gate") that runs overnight on one 16 GB M1 MacBook, starting at 10 pm Mountain on November 2, 2026.
A pass justifies the next stage: more seeds, larger models, real hardware. A fail should say what broke, so every gate includes a positive control or a headroom check that separates "the idea failed" from "the pipeline failed".
Our previous note on per-token layer routing is why the rules are written down first. There, a hindsight oracle suggested large gains, and most of that gain turned out to be selection. Fixing the metric, sample size and pass rule before the data exist is the cheapest protection we know against repeating that.
The four articles
| Article | Question | Pass rule |
|---|---|---|
| Can a model sleep its way to grokking sooner? | Does distilling a model toward its own generalizing circuit speed up grokking more than tuned weight decay and matched shams? | A perfect-information version beats the best weight decay (p < 0.05), and no sham or generic control explains the gain |
| Is there a reason for a learning rule to develop? | Does a task reward tuning a learning rule in "childhood", and could we detect and find such a rule? | Real headroom, a detectable built-in effect, and an optimizer that recovers ≥ 50% of it |
| Can you score a learning rule without labels? | Do label-free scores rank self-supervised learning rules the way a linear probe does? | Spearman ρ ≥ 0.5 with the 95% CI lower bound > 0.3 |
| Where does backprop clearly win? | Where do learning rules without backprop's weight transport fall ≥ 10 points behind it? | Reports the cheapest architecture with that gap, or none |
Shared protocol
- Each project's rules are frozen in a
SPEC.md. Any change goes under a dated "Amendments" heading with the reason. Analyses not in the spec are reported as exploratory. - No seeds or arms are added to push a result over a threshold.
- Every verdict is reported exactly as the pre-registered analysis script writes it, including fails.
- The code is not public yet. Each article lists the SHA-256 hash and commit of its frozen spec, so the timing can be checked when the code is released.
- Amendment, November 2, 2026: the run started at 7:58 pm Mountain, not 10 pm as stated above. The launcher scheduled for 10 pm had stopped, so the run was started by hand. All five pages were published before the run started, and no arm, metric, sample size or pass rule changed.