The generator gets rewarded when the solver fails. Its reward went up, but its puzzles got easier. Below, two versions of that training run side by side in your browser.
Both panes train on the same puzzles with the same reward. The right one adds two fixes: an entropy bonus and a penalty for duplicate puzzles. Press run and watch the left one stop learning.
What this is showing is the zero-variance trap, and that part is arithmetic rather than something I fitted: when every sample in a group is the same and the reward is saturated, the group-relative advantage is exactly zero, so the policy cannot move no matter what the learning rate is. Difficulty and solvability here are exact. The solver's pass rate is a stand-in, because a 0.5B language model will not run in a browser. It is calibrated against measured probe bins for the difficulty slope, the audited generator-versus-control gaps for the small-number effect, and the measured within-bin pass variance of 0.1397 for how much puzzles of the same difficulty actually differ. The separate finding about distribution shift comes from the real runs and is in section 03, not here.
Try it: on monotone hardness the original collapses onto a single puzzle the solver never solves, its reward pins at 1.0, and roughly half its updates die. Switch to edge-seeking and it still collapses onto one proposal, but the reward stays noisy so the gradient survives. Collapse and gradient death turn out to be two different failures.
Three numbers, one target, four operators. There are exactly 108 ways to combine them. This page enumerates all of them in exact rational arithmetic and counts how many reach the target. That count is the ground truth everything else in the study gets measured against. Type in anything you want and watch it work.
Each tick on the left is one of the 108 combination paths, ordered by the value it produces. Lit ticks reach your target. Difficulty is −log₂ of the lit fraction, so a puzzle with one solution out of 108 scores 6.75 bits and one with twenty scores 2.43.