Skip to content
The loss curve

Out-of-vocabulary

A token (or pair) that never appeared during training. Unsmoothed models can't assign it any probability.

Companion explanation in Step by Token, chapter 2.