Temperature
A divisor applied to logits before softmax. Below 1 sharpens the distribution (safer, more repetitive); above 1 flattens it (more surprising, less coherent).
Temperature 0 is the limit case: the distribution collapses onto its maximum and sampling becomes argmax. It is the single most useful generation knob, and the first one to reach for when output is either too bland or too unhinged.
Companion explanation in Step by Token, chapter 7.
Where this term gets built
Continue