Research article · Pre-registration
Is there a reason for a learning rule to develop?
Results are in: Results: is there a reason for a learning rule to develop?
Abstract
Animals tune how they learn to the world they grow up in. We want to evolve learning rules that do the same: set their own parameters from early experience. Before building that in neural networks, we check three prerequisites on a simple reinforcement-learning agent whose one developmental parameter is τ, how long credit for a choice lingers when reward arrives late. Is there headroom, meaning does the best τ depend on the environment enough to matter? Can our analysis detect a developmental effect we build in by hand? Can a simple evolution strategy find that hand-built rule? The decision rules are fixed here, before any full-run result exists.
1. Introduction
Learning-to-learn usually fixes a learning rule's parameters across a lifetime. Development is a different idea: an agent spends a "childhood" estimating something about its environment, then sets how it learns as an adult. If that is useful, evolved rules should discover it.
Three things have to hold before evolving such rules is worth doing. The environment must give development something to do. Our analysis must be able to see a developmental effect when one exists. And the optimizer must be able to find one. Each gets its own check below, so a null result can be traced to the step that failed.
This is a gate, not a paper. A pass justifies moving to neural networks.
2. Methods
Agent. Leaky action preferences, a softmax policy (β = 6), a replacing eligibility trace with timescale τ, and a reward baseline.
Environment. The reward for the choice made at step t − d arrives at step t, and the agent isn't told which choice earned it. The good option reverses once in mid-adulthood. Childhood is 300 trials and adulthood is 300 trials. Each condition is averaged over 2,000 lifetimes.
Tasks. A plain two-armed bandit (C1) and a contextual bandit with four cues (C4).
3. Checks and decision rules
Headroom. A grid over τ and learning rate η for reward delays of 0 and 3, in both tasks. The cost of a wrong τ is the reward lost by using the other delay's best τ. PASS: both costs have a 95% CI excluding 0, and the larger is ≥ 0.05.
Positive control. A hand-built developmental rule estimates the delay during childhood from lagged correlations and sets τ from the headroom lookup. Twin A is raised at delay 0 and twin B at delay 3, then they are tested as adults in both. We measure the adult crossover interaction, the effect of transplanting τ between twins, and the effect of transplanting their learned weights, against a rule with fixed τ. PASS: the crossover CI is above 0, the τ-transplant CI is above 0, and the fixed rule's crossover is less than a third of the built-in rule's.
Optimizer check. A mirrored evolution strategy (population 16, 40 generations) over τ = exp(w₀ + w₁·d̂), where d̂ is the delay estimated in childhood. PASS: it recovers at least 50% of the reward gap between the best fixed τ and the built-in rule.
4. Observations made while building (not verdicts)
In a prototype run, the plain two-armed bandit failed the headroom check: choices are sticky, so a short trace works at any delay and development has nothing to do. The contextual bandit passed. If that holds tonight, the research task moves to the contextual version.
5. Protocol and amendments
- Rules are frozen in the project's
SPEC.md. Any change goes under a dated "Amendments" heading with the reason. Analyses not in the spec are reported as exploratory. - No lifetimes or conditions are added to push a result over a threshold.
- Amendments so far, both before the full run and neither changing the spec's rules: the code was moved from a combined bundle into its own project, and a smoke test passed (3 of 3 jobs, no errors).
- Amendment, November 2, 2026: the run started at 7:58 pm Mountain, not 10 pm as first planned. The launcher scheduled for 10 pm had stopped, so the run was started by hand. This article was published before the run started, and no arm, metric, sample size or pass rule changed.
6. Verifying this pre-registration
The code is not public yet. The frozen spec has SHA-256 hash db1a9f1957ea32c9e0122a5675f0ae0ff7dc8a622d395069e4dd6173db67771d, committed November 2, 2026 at 11:08 Mountain in commit d799df8. When the code is released, anyone can confirm the spec matches.
7. Limitations
- A tabular agent, not a network. This checks that the question is well-posed and measurable, not that neural networks will evolve development.
- One developmental parameter. Only τ adapts. Real development changes many things at once.
- The positive control is built by hand, so it shows the analysis can detect development, not that development is the best strategy.