CacheGen

LMCache is a KV-cache management layer for AI inference systems. It stores pages of the KV cache from GPU memory in remote storage and provides an integration point for reading stored KV pages, including for security analysis and threat discovery.

NVIDIA Dynamo is a developer toolkit from NVIDIA for managing distributed key–value (KV) context caches used in large‑language‑model inference. It provides KV‑aware routing, cache offloading, and prefill/decode disaggregation to move KV cache and related activation/state across GPU clusters, reducing memory pressure and supporting scalable LLM serving. It is intended for integration into GPU‑based inference and serving stacks.

Intrinsec AI is an AI-inference security company developing products that use KV-cache data as a security signal for production LLM inference. It learns the normal computational shape of a workload from its inference traffic and surfaces unusual KV pages, rather than inspecting prompts or relying on a separate detector for each attack or failure. Its Tessera product analyzes KV cache at rest, while Plato acts on anomalous KV data in flight; the system is intended to cover text, voice, tool-using, and agent workloads and can integrate with inference engines or KV-cache management layers.

Gimlet Cloud is Gimlet Labs' serverless inference platform for AI agents and multi-agent systems. It can run agentic pipelines with custom logic, data sources, chained models, and non-model stages such as search; the platform manages scheduling, orchestration, and optimization. Its multi-silicon approach splits inference workloads into fine-grained pieces and maps them to suitable hardware, including Nvidia, AMD, AWS Trainium, Cerebras, D-Matrix, and Intel Gaudi systems. Gimlet describes this heterogeneous execution model as achieving 3–10× faster inference than homogeneous systems. The company also develops kforge, a multi-agent system that generates and checks optimized low-level kernels from PyTorch for CUDA, ROCm, and Metal backends.

Gimlet Labs is an applied research lab and AI-infrastructure company building systems for serving AI workloads. Its Gimlet Cloud provides serverless inference for AI agents, supporting multi-agent systems, custom logic and data sources, model chains with non-model stages such as search, and platform-managed scheduling, orchestration and optimization. The company describes a multi-silicon inference architecture that divides workloads into fine-grained pieces and maps them to suitable hardware, including GPUs and specialized accelerators. Its research also covers workload partitioning, heterogeneous-hardware compilation, dynamic datacenter scheduling, hybrid edge/cloud execution and cost-aware optimization. KForge is its multi-agent system for generating optimized low-level kernels directly from PyTorch, with correctness checks and support for CUDA, ROCm and Metal backends.