article
A looped-layer experiment with two checks to kill it, and a training stage that can't learn
One copied layer looped in place of a stretch of middle layers, two cheap scripts meant to rule it out before training, the places those scripts are softer than they look, and a frozen tail that blocks the second stage's gradient.
Read more