Skip to content
The loss curve

Logits

The raw, unnormalized scores a model emits — one per vocabulary token. Softmax turns them into probabilities; before that they can be any real number.

A model's final layer produces one logit per token in the vocabulary, for every position. Nothing about them is a probability yet: they can be negative, and they do not sum to 1. Sampling operates on logits (temperature divides them) and on the probabilities that follow (top-k, top-p filter those).

Companion explanation in Step by Token, chapter 7.

Where this term gets built