Research article · Results

Results: can you score a learning rule without labels?

Kāula research note, November 3, 2026. Results for the pre-registered gate in Can you score a learning rule without labels?. Every number below comes from the pre-registered analysis script and its raw results.

Verdict

Gate: PASS (pre-registered: rho >= 0.5 with CI lower bound > 0.3). Best label-free score: RankMe.

That's the analysis script's verdict line, exactly as written. RankMe passes. Across 108 rules, its Spearman correlation with linear-probe accuracy is 0.583, with a 95% CI of 0.443 to 0.695. LiDAR and cross-seed agreement don't pass.

As pre-registered, a pass picks the fitness score for the search: RankMe is the label-free score to evolve self-supervised local learning rules against.

1. Results

All 108 rules finished, with no errors: 8 known rules (including the untrained random-initialization network) and 100 random rules. Each label-free score is compared with linear-probe accuracy across the 108 rules.

Label-free scoreSpearman ρ vs probe accuracy95% CIRules
LiDAR0.3410.162 to 0.498108
RankMe0.5830.443 to 0.695108
Cross-seed kNN agreement, excess over random init−0.242−0.412 to −0.055108
kNN accuracy (uses labels; reference only)0.9300.900 to 0.952108

The pass rule needs ρ ≥ 0.5 and a CI lower bound above 0.3. RankMe clears both. LiDAR's lower bound is above zero but its ρ is below 0.5. Cross-seed agreement is negatively correlated with probe accuracy: rules whose two seeds agree more tend to probe worse.

kNN accuracy uses labels, so it can't be a fitness score. It's reported as a check that the probe is measuring something stable: two label-based measures of the same representations agree closely (ρ = 0.930).

2. What else the report shows

These observations weren't part of the pre-registered rule, so they are exploratory:

3. What the verdict means

The search can use RankMe as its fitness score without labels. That's what the gate was for: an evolutionary search optimizes whatever it's given, and of the three scores tested, only RankMe ranks rules in roughly the order a linear probe does.

It doesn't mean RankMe can't be gamed. A correlation of 0.58 leaves a lot of room for a search to find rules that score well and probe badly, so the search should keep checking its best rules against a probe.

4. Deviations from the plan

5. Limitations

What's next

Evolve self-supervised local learning rules against RankMe, and check the best rules found against a linear probe as the search goes.

All notes · The four pre-registrations