AI use cases

Reuse previously computed attention keys and values during autoregressive generation so each new token computes only its own query, key, and value before attending over the cached history.

From How KV Cache Speeds Up LLMs for Faster AI Models on GPUs by IBM Technology