Research article · Pre-registration

Is there a reason for a learning rule to develop?

Kāula research note, November 2, 2026. Pre-registration: the metrics, sample sizes and pass rules below were fixed before tonight's run. Results follow in a companion note.

Results are in: Results: is there a reason for a learning rule to develop?

Abstract

Animals tune how they learn to the world they grow up in. We want to evolve learning rules that do the same: set their own parameters from early experience. Before building that in neural networks, we check three prerequisites on a simple reinforcement-learning agent whose one developmental parameter is τ, how long credit for a choice lingers when reward arrives late. Is there headroom, meaning does the best τ depend on the environment enough to matter? Can our analysis detect a developmental effect we build in by hand? Can a simple evolution strategy find that hand-built rule? The decision rules are fixed here, before any full-run result exists.

1. Introduction

Learning-to-learn usually fixes a learning rule's parameters across a lifetime. Development is a different idea: an agent spends a "childhood" estimating something about its environment, then sets how it learns as an adult. If that is useful, evolved rules should discover it.

Three things have to hold before evolving such rules is worth doing. The environment must give development something to do. Our analysis must be able to see a developmental effect when one exists. And the optimizer must be able to find one. Each gets its own check below, so a null result can be traced to the step that failed.

This is a gate, not a paper. A pass justifies moving to neural networks.

2. Methods

Agent. Leaky action preferences, a softmax policy (β = 6), a replacing eligibility trace with timescale τ, and a reward baseline.

Environment. The reward for the choice made at step t − d arrives at step t, and the agent isn't told which choice earned it. The good option reverses once in mid-adulthood. Childhood is 300 trials and adulthood is 300 trials. Each condition is averaged over 2,000 lifetimes.

Tasks. A plain two-armed bandit (C1) and a contextual bandit with four cues (C4).

3. Checks and decision rules

Headroom. A grid over τ and learning rate η for reward delays of 0 and 3, in both tasks. The cost of a wrong τ is the reward lost by using the other delay's best τ. PASS: both costs have a 95% CI excluding 0, and the larger is ≥ 0.05.

Positive control. A hand-built developmental rule estimates the delay during childhood from lagged correlations and sets τ from the headroom lookup. Twin A is raised at delay 0 and twin B at delay 3, then they are tested as adults in both. We measure the adult crossover interaction, the effect of transplanting τ between twins, and the effect of transplanting their learned weights, against a rule with fixed τ. PASS: the crossover CI is above 0, the τ-transplant CI is above 0, and the fixed rule's crossover is less than a third of the built-in rule's.

Optimizer check. A mirrored evolution strategy (population 16, 40 generations) over τ = exp(w₀ + w₁·d̂), where d̂ is the delay estimated in childhood. PASS: it recovers at least 50% of the reward gap between the best fixed τ and the built-in rule.

4. Observations made while building (not verdicts)

In a prototype run, the plain two-armed bandit failed the headroom check: choices are sticky, so a short trace works at any delay and development has nothing to do. The contextual bandit passed. If that holds tonight, the research task moves to the contextual version.

5. Protocol and amendments

6. Verifying this pre-registration

The code is not public yet. The frozen spec has SHA-256 hash db1a9f1957ea32c9e0122a5675f0ae0ff7dc8a622d395069e4dd6173db67771d, committed November 2, 2026 at 11:08 Mountain in commit d799df8. When the code is released, anyone can confirm the spec matches.

7. Limitations

All notes · The four pre-registrations