Research article · Pre-registration
Can you score a learning rule without labels?
Results are in: Results: can you score a learning rule without labels?
Abstract
To evolve self-supervised local learning rules, we need a fitness score that doesn't use labels. Before evolving against one, we check that it ranks rules the way a labelled linear probe does. We train 108 local learning rules (8 known rules and 100 random mixtures of 8 local terms) on two augmented views of CIFAR-10 images, with 2 seeds each, and compare three label-free scores, LiDAR, RankMe and cross-seed nearest-neighbour agreement, with linear-probe accuracy across rules. A score passes if its Spearman correlation with probe accuracy is at least 0.5 with a 95% CI lower bound above 0.3. The decision rule is fixed here, before any full-run result exists.
1. Introduction
Evolutionary search optimizes whatever score it is given. If a label-free score only loosely tracks representation quality, a search will find rules that game the score. So the first question isn't which rule is best, but whether any cheap label-free score can tell good rules from bad ones across the kind of population a search would generate.
This is a gate, not a paper. A pass picks the fitness score for the search. A fail means we don't evolve against these scores.
2. Methods
Rules. 108 rules: 8 known rules and 100 random unit vectors over an 8-term local basis: Hebbian, cross-view Hebbian, predictive (pull the two views together), Oja decay, decorrelation, variance-normalized Hebbian, weight decay and covariance Hebbian.
Network and training. A two-layer ReLU network (3,072 → 512 → 256), with weight rows renormalized to unit length after each update. Each image gets two augmented views (crop, flip, brightness and contrast, grayscale). Training is 1,500 steps at batch size 256 on 20,000 CIFAR-10 training images, with 2 seeds per rule.
Label-free scores.
- LiDAR (Thilak et al., 2024): 500 images × 8 views.
- RankMe (Garrido et al., 2023): 2,048 test images.
- Cross-seed agreement: mutual k-nearest-neighbour overlap (k = 10, 1,000 images) between the two seeds' representations, minus its value at random initialization.
Reference. Linear-probe accuracy (trained on 10,000 features, tested on 5,000), with kNN accuracy reported alongside.
3. Decision rule
The primary test is the Spearman correlation ρ between each label-free score and probe accuracy, across rules.
- PASS if some score has ρ ≥ 0.5 with a 95% CI lower bound above 0.3.
- FAIL otherwise: don't evolve against these scores.
4. Protocol and amendments
- Rules are frozen in the project's
SPEC.md. Any change goes under a dated "Amendments" heading with the reason. Analyses not in the spec are reported as exploratory. - The spec allowed lowering the number of random rules to no fewer than 60 if time ran short. A timing benchmark on the M1 (about 1.1 minutes per rule) showed all 100 fit, so the full pre-registered count runs.
- Another amendment, before the full run: the code was moved from a combined bundle into its own project.
- Amendment, November 2, 2026: the run started at 7:58 pm Mountain, not 10 pm as first planned. The launcher scheduled for 10 pm had stopped, so the run was started by hand. This article was published before the run started, and no arm, metric, sample size or pass rule changed.
5. Verifying this pre-registration
The code is not public yet. The frozen spec has SHA-256 hash ede96bda659ce11d264ddefe4c7cc0eb1c926a331c553c1eafff9f725fa92b56, committed November 2, 2026 at 11:08 Mountain in commit 490d933. When the code is released, anyone can confirm the spec matches.
6. Limitations
- Small networks, short training. A score that works here may not track quality in deeper networks or longer runs.
- Two seeds per rule. Enough for a correlation across 108 rules, but each rule's own accuracy is noisy.
- A linear probe is one view of quality. We treat it as the reference, but it isn't the only one.
References
- Garrido, Q., Balestriero, R., Najman, L. & LeCun, Y. (2023). RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank. ICML.
- Thilak, V., Huang, C., Saremi, O., Dinh, L., Goh, H., Nakkiran, P., Susskind, J. & Littwin, E. (2024). LiDAR: Sensing linear probing performance in joint embedding SSL architectures. ICLR.