Runpod

Runpod Flash is an open-source Python SDK and application framework from Runpod for developing cloud-native AI applications and running serverless GPU and CPU workloads without manually writing Dockerfiles or building and pushing container images. Developers define remote functions with Python decorators, specifying hardware, dependencies, workers, and concurrency, then use `flash deploy` to package code, build cross-platform worker artifacts, install dependencies, and provision endpoints. Its `flash dev` mode runs a local FastAPI server while provisioning live ephemeral Runpod workers. Flash supports queue-based endpoints for asynchronous and batch workloads, load-balanced endpoints with HTTP routes, custom Docker images, endpoint-to-endpoint calls, connections to existing Runpod endpoints by ID, automatic dependency installation, and worker scaling from zero to the configured maximum. It supports multi-endpoint applications in which functions on different endpoints call one another through service discovery and routing handled by the runtime. It requires Python 3.10 or later on macOS or Linux.

Runpod Flash Examples is a GitHub repository of example applications for Runpod Flash, a Python framework for running functions on Runpod Serverless infrastructure with distributed GPU and CPU computing. The examples cover GPU, CPU, and mixed workers, dependency management, local modules, model serving, load-balanced HTTP routes, autoscaling, and network volumes. The repository demonstrates the Flash workflow, including the `@Endpoint` decorator, local development with hot reload, deployment with `flash build` and `flash deploy`, and endpoint-to-endpoint client calls. It is maintained as an open-source example collection with contribution and development guides, and the README indicates that additional examples are planned. The repository is licensed under MIT.

Runpod's documentation site for Flash, a Python SDK for developing cloud-native AI applications on Runpod. It covers setup, authentication, configuration, deployment, usage, the Flash CLI, tutorials, examples, and limitations. The documentation explains how locally written Python functions decorated with `@Endpoint` can define remote hardware, dependencies, and execution, while Flash provisions GPU or CPU resources and scales workers on Runpod Serverless. It also documents Flash apps, which are collections of interconnected endpoints that can be deployed as production APIs. The site was last modified on July 20, 2026.

BenchPod is a Runpod benchmarking dashboard and template built on vLLM. It packages vLLM in a Docker image with a FastAPI layer, centralizes GPU statistics, cold-start measurements, inter-token latency, time to first token, and time per output token, and allows vLLM configurations to be changed and tested live without waiting for restarts. It provides OpenAI-compatible endpoints for agents and includes a watchdog intended to recover failed instances. The video describes it as not yet released and expected to be distributed as a Runpod template.

Cloud Run is Google Cloud's fully managed serverless platform for deploying and running containerized applications and services. It automatically scales instances with incoming traffic, can scale to zero when idle, and provides load balancing, TLS, logging, and identity management.