Research article · Results
Results: is there a reason for a learning rule to develop?
Verdict
All three pre-registered checks pass, but only on the contextual task.
- Headroom: PASS on the contextual bandit with four cues. The plain two-armed bandit fails, as the prototype suggested.
- Positive control: PASS. The analysis detects a built-in developmental effect, and attributes it to τ, not to the learned weights.
- Optimizer check: PASS. A simple evolution strategy recovers 93% of the gap between the best fixed τ and the built-in rule.
Following the pre-registered plan, the research task moves from the plain bandit to the contextual one.
1. Headroom
The cost of a wrong τ is the reward lost by using the other delay's best τ. 2,000 lifetimes per condition; 95% bootstrap CIs.
| Task | Best τ, delay 0 | Best τ, delay 3 | Cost at delay 0 | Cost at delay 3 | Pass |
|---|---|---|---|---|---|
| Plain two-armed bandit | 0.25 | 4 | 0.006 [0.006, 0.006] | 0.005 [0.002, 0.009] | No |
| Contextual bandit, 4 cues | 1 | 12 | 0.045 [0.044, 0.047] | 0.151 [0.146, 0.156] | Yes |
The rule needs both costs to exclude 0 and the larger to be at least 0.05. In the plain bandit both costs are real but tiny, at about half a percentage point of reward: choices are sticky, so a short trace works at any delay. In the contextual bandit, the wrong τ costs 15 points of reward at delay 3. One τ cannot serve both environments.
2. Positive control
A hand-built developmental rule estimates the delay during childhood and sets τ from the headroom lookup. On the contextual task it estimated the delay correctly in every lifetime, choosing τ = 1 for twins raised at delay 0 and τ = 12 for twins raised at delay 3.
| Measure | Developmental rule | Fixed τ = 6 |
|---|---|---|
| Adult crossover interaction | 0.192 [0.186, 0.198] | 0.000 [−0.006, 0.006] |
| τ-transplant effect | 0.196 [0.192, 0.199] | 0 |
| Weight-transplant effect | −0.004 [−0.009, 0.001] | 0.000 |
All three pass conditions hold: the crossover CI is above 0, the τ-transplant CI is above 0, and the fixed rule's crossover is far below a third of the developmental rule's. Transplanting τ carries the whole effect; transplanting the learned weights carries none. So the analysis can tell a developmental change in how an agent learns from a difference in what it learned.
3. Optimizer check
A mirrored evolution strategy (population 16, 40 generations) searched over τ = exp(w₀ + w₁·d̂), where d̂ is the delay estimated in childhood.
| Rule | Adult fitness |
|---|---|
| Best fixed τ | 0.727 |
| Evolved rule | 0.753 |
| Built-in rule | 0.755 |
The evolved rule recovers 93% of the gap (pass: ≥ 50%). It settled on τ ≈ 0.28 at delay 0 and τ ≈ 14.4 at delay 3, close in spirit to the built-in rule's 1 and 12.
4. Deviations from the plan
- The run started at 7:58 pm Mountain, not 10 pm, as recorded in the pre-registration's amendments. All three jobs finished by 7:59 pm.
- The run was later paused at 8:32 pm because other work saturated the Mac. This gate had already finished, so the pause didn't affect it. Machine load doesn't affect these results in any case: they are counted in trials, not time.
- No other changes. The verdicts are the script's.
5. Limitations
- A tabular agent. This shows the question is well-posed and measurable on a toy task, not that neural networks will evolve development.
- The optimizer search space is small. Two parameters, with the delay estimate handed to the rule. A full search would have to discover what to estimate in childhood too.
- One developmental parameter. Only τ adapts.
What's next
The developmental learning-to-learn project uses the contextual bandit as its task, and moves on to evolving rules that must discover for themselves what to measure in childhood.