Softmax
Normalizes a vector of real numbers into a probability distribution: exp(x_i) / Σ exp(x_j). Used at the end of attention and at the model's output.
Companion explanation in Step by Token, chapter 4.
Where this term gets built
Continue
Normalizes a vector of real numbers into a probability distribution: exp(x_i) / Σ exp(x_j). Used at the end of attention and at the model's output.
Companion explanation in Step by Token, chapter 4.
Continue