Prefill
The first pass, where the whole prompt is processed at once to populate the KV cache. It determines time-to-first-token; the per-token decoding that follows is much cheaper.
Companion explanation in Step by Token, chapter 18.
Continue
The first pass, where the whole prompt is processed at once to populate the KV cache. It determines time-to-first-token; the per-token decoding that follows is much cheaper.
Companion explanation in Step by Token, chapter 18.
Continue