LLM

Generating token N means attending to tokens 1..N-1. Recomputing their key/value tensors every step turns O(n) work into O(n²). Caching K and V is what makes real-time LLM streaming possible.