Top-p (nucleus) sampling
Keep the smallest set of tokens whose probabilities sum to p, renormalize, then sample. Unlike top-k, the size of that set adapts to how confident the model is.
Companion explanation in Step by Token, chapter 7.
Where this term gets built
Continue