Batch
A group of training examples processed together in one forward and backward pass. Bigger batches give a less noisy gradient and use hardware better, at the cost of memory.
Companion explanation in Step by Token, chapter 6.
Where this term gets built
Continue