Continual learning research · measured result · November 2026

A third of forgetting is the arbitrary choice in the example

With the content of every demonstration held perfectly fixed, the convention alone moves forgetting by a third, and a skill learned from all equally good answers at once has no convention to lose.

Measured result · November 2026

The arbitrary choice in a demonstration is a third of what gets forgotten

Most tasks have several equally right answers, and a demonstration has to show one of them. That pick, the convention, is not content: any of the answers would do. In grid worlds where an exact solver knows every best move, I could hold the content of every demonstration perfectly fixed and vary only the convention. Teaching a second task with demonstrations that break ties the opposite way from the first task’s costs a third of the first skill, with identical content. A skill learned from all the right answers at once has no convention to lose, and keeps that third.

1,088 training runs over two days: 624 on one rented GPU for the main grid (6 task pairs, 26 ways of teaching, 4 seeds), 136 follow-ups, 328 on a CPU for a synthetic model. Predictions were written down before the sweep. Small networks and small worlds; not yet tested on language models.

Three numbers

Retention is the first skill’s success on new levels during second-task training, relative to before it. Paired over task pair and seed, 24 cells.
Comparison	Retention	Cells with the same sign
Second task taught with the opposite tie-break instead of the same one (identical content)	−0.10	24 of 24
First skill learned from all optimal actions at once instead of one canonical path, opposite tie-break later	+0.10	24 of 24
The same all-answers skill when the two tasks happen to share a tie-break	±0.00	8 of 24

Three bar charts: retention of the first task by how it was learned, by how the second task is taught, and per task pair canonical versus all-answers — Left: how the first skill was learned (c1 and c2 one canonical path each with opposite tie-breaks, rand one random optimal path per level, ls label smoothing, soft and dag all optimal actions at once). Middle: how the second task is taught when the first used convention c1: same convention 0.80, opposite 0.70, the path the learner already prefers 0.80. Right: canonical against all-answers for each of six task pairs.

Forgetting curves for one task pair: first-task success against second-task training steps, one line per way of teaching — One pair in detail (open rooms, then keys): first-task success against second-task steps, log scale. Left, the first skill’s encoding; the all-answers encodings stay highest. Right, the second task’s demonstrations; the opposite convention (red) hurts most and the learner-preferred path (orange) matches the same-convention line.

How the test works, and what else it showed

Exact demonstrations

Long Game levels (keys, gates, open rooms) have a solver that finds every reachable state and its best moves. Every demonstration is provably optimal; the training arms differ only in which equally good move they show, or whether they show all of them. A small network learns one task for 6000 steps, then another for 3000 with no data from the first, and the first task is measured exactly 28 times along the way.

Learner-preferred demonstrations

Re-choosing, among optimal paths, the one the current policy already prefers recovers exactly the convention effect (+0.10, 24 of 24). A controlled reason why on-policy data forgets less (RL’s Razor, ICLR 2026; Self-Distillation Fine-Tuning, 2026): it cannot clash with the learner’s conventions.

Distance between policies is the wrong yardstick

At the same KL divergence from the original policy, a skill without conventions loses about half as much success as a canonical one. For a canonical skill, over half of its measured shift is harmless drift among equally good moves.

Checks

Holds with the optimizer state kept across tasks (+0.13), a 3× higher learning rate (+0.12) and a 2× wider network (+0.12). Memorised levels forget as fast as new ones. Heavy label smoothing (0.3–0.5) also removes most of the cost in the grid worlds, but not in the synthetic model below. Refining the canonical skill with on-policy reinforcement learning leaves its convention in place and gains nothing unless pushed with a strong entropy bonus, which costs accuracy.

Practical reading: when several answers are correct, train the skill you want to keep on all of them, or on the learner’s own preferred ones; when fine-tuning later, prefer demonstrations that match the model’s existing conventions.

The same signature in a synthetic model

A small classifier where each input cluster has several correct labels, and a second task that shares the output layer. The gap between the canonical and the all-answers encoding is zero when no input has equivalents and 15 points when every input does; a clashing convention costs 9 points; label smoothing hurts here.

Synthetic model bar charts: the canonical encoding loses more when the second task uses the opposite tie-break; the gap holds for 2, 3 and 5 equivalent labels and for narrow and wide networks — (a) The canonical encoding keeps 0.76 when the next task shares its tie-break and 0.67 when it opposes it; random-per-cluster and all-correct encodings do not care. (b, c) With equivalents on every input and an opposing convention, the gap is 20–28 points for 2, 3 or 5 equivalents and for narrow or wide networks.

Synthetic model forgetting curves at 0, 25, 50 and 100 percent of inputs with equivalents — The gap opens with the share of inputs that have equivalents. Label smoothing (brown) is worse everywhere.

Code, pre-registered protocol and the full report with every number: research-lab/experiments/exact-oracle-forgetting (report.md). One RTX 4090 for 4.8 hours, about $4. A rough draft of the paper, with the closest prior work, is at /paper/conventions-forgetting/.

More research