Skip to content
The loss curve

Context window

The maximum number of tokens a model can attend over at once. Everything outside it is invisible to the model — there is no memory beyond the window.

Companion explanation in Step by Token, chapter 9.

Where this term gets built