Research article · Pre-registration
Where does backprop clearly win?
Results are in: Results: where does backprop clearly win?
Abstract
Searching for new learning rules only makes sense on a network where existing alternatives to backprop clearly lose. On shallow networks they come close, so a search there would return a guaranteed null. We train backprop, feedback alignment and sign-symmetry on CIFAR-10 across an MLP and convnets of 3, 5 and 7 layers, and report the cheapest architecture where backprop beats the best alternative by at least 10 points. That architecture becomes the testbed for rule search. If none qualifies, the search waits for larger hardware. The decision rule is fixed here, before any full-run result exists.
1. Introduction
Backprop sends errors backward through the transpose of the forward weights, which is hard to square with biology. Feedback alignment (Lillicrap et al., 2016) replaces the transpose with fixed random weights. Sign-symmetry (Liao et al., 2016) keeps only the signs. Both work surprisingly well on small networks and fall behind as networks get deeper.
A search for better rules needs a gap to close. If the testbed has no gap, every candidate looks as good as backprop and the search learns nothing. This gate finds the smallest network that has a clear gap.
2. Methods
Learning rules. Backprop, feedback alignment (fixed random feedback weights) and sign-symmetry.
Architectures. An MLP with three hidden layers of 1,024 units, and convnets with 3, 5 or 7 convolutional layers (64 channels, batch normalization).
Training. 25,000 CIFAR-10 training images, 4 epochs, Adam at learning rate 10⁻³, 2 seeds. Tested on the 10,000-image test set.
3. Decision rule
Output: the cheapest architecture where backprop's test accuracy minus the better of feedback alignment and sign-symmetry is at least 10 points. That architecture is the proxy testbed for the rule search. If none qualifies, there is no headroom at this budget.
4. Protocol and amendments
- Rules are frozen in the project's
SPEC.md. Any change goes under a dated "Amendments" heading with the reason. Analyses not in the spec are reported as exploratory. - No architectures or seeds are added after the run to manufacture a gap.
- Amendments so far, both before the full run and neither changing an arm, seed or the rule: the code was moved from a combined bundle into its own project and now runs first on the GPU, and a timing benchmark on the M1 put all 24 jobs at about 0.3 hours.
- Amendment, November 2, 2026: the run started at 7:58 pm Mountain, not 10 pm as first planned. The launcher scheduled for 10 pm had stopped, so the run was started by hand. This article was published before the run started, and no arm, metric, sample size or pass rule changed.
5. Verifying this pre-registration
The code is not public yet. The frozen spec has SHA-256 hash b4804c78d87f4a42db7b41b757ecab97e3cc1bfa6682c5782633019cb4024042, committed November 2, 2026 at 11:08 Mountain in commit fdffa41. When the code is released, anyone can confirm the spec matches.
6. Limitations
- Short training. Four epochs favours whichever rule learns fastest early. Gaps may change with longer training.
- Two seeds. Enough to see a 10-point gap, not to measure small ones.
- Two alternatives. Other backprop-free rules, such as target propagation or direct feedback alignment, aren't included.
References
- Liao, Q., Leibo, J. Z. & Poggio, T. (2016). How important is weight symmetry in backpropagation? AAAI.
- Lillicrap, T. P., Cownden, D., Tweed, D. B. & Akerman, C. J. (2016). Random synaptic feedback weights support error backpropagation for deep learning. Nature Communications.