Gradient
Vector of partial derivatives of the loss with respect to every parameter. Tells you which way (and how much) to nudge each parameter to lower the loss.
Companion explanation in Step by Token, chapter 6.
Where this term gets built
Continue
Vector of partial derivatives of the loss with respect to every parameter. Tells you which way (and how much) to nudge each parameter to lower the loss.
Companion explanation in Step by Token, chapter 6.
Continue