AI product
TensorRT is NVIDIA's inference-serving runtime for optimizing models for high throughput. In the cited context, it is used as a serving-runtime option when deciding whether to run inference on dedicated GPUs, alongside measurements of latency, throughput, precision, memory use, and workload performance.
1 use taken from transcripts — each links to the moment in the video.
Nvidia serving runtime for peak throughput
1 in the library.