AI use cases
Generate tokens from user prompts across prefill and decode phases.
From How KV Cache Speeds Up LLMs for Faster AI Models on GPUs by IBM Technology