Skip to content
The loss curve

One-hot encoding

A vector of zeros with a single 1 at the position of the token. Wasteful and asserts that every pair of distinct tokens is equally unrelated.

Companion explanation in Step by Token, chapter 3.

Where this term gets built