FreeLLMAPI

LiteLLM is an open-source AI gateway and Python SDK from BerriAI for calling more than 100 LLM providers through a unified OpenAI-compatible interface. It supports direct library integration and a self-hosted proxy server, with provider-specific formats translated into common endpoints such as chat completions, responses, embeddings, image generation, audio, batches, reranking, messages, and agent-to-agent calls. The proxy provides centralized authentication and authorization, virtual keys, per-project and per-user spend tracking, logging, guardrails, caching, load balancing, routing with retry and fallback logic, and an administration dashboard. LiteLLM also exposes MCP tools and A2A agents through the gateway and can be deployed as a componentized stack with separate gateway, backend, and UI services. The repository describes it as open source and documents Docker, local development, and Terraform deployment options for AWS and Google Cloud.

RouteLLM is an open-source framework from lm-sys for serving and evaluating routers that direct requests between stronger and cheaper language models. It provides trained routers, a drop-in replacement for the OpenAI client, and an OpenAI-compatible server; routing decisions use configurable cost thresholds that can be calibrated for a target share of requests sent to the stronger model. The framework supports comparing routers across benchmarks and extending it with additional routers, model pairs, and providers.

Experiential Labs is an open-source AI gateway that provides an OpenAI-compatible endpoint for hosted model providers, customer-owned provider keys, private GPUs, and self-hosted models. It routes requests through a single API and key while handling provider access, model selection, failover, and streaming responses. Its optional intelligence layer monitors traffic to identify model switches, improve cache hit rates, and route requests to customer-owned fine-tuned models. The service also provides organization-wide usage attribution, request logs, spend reporting, access controls, model allowlists, and per-key spending caps scoped by team, person, agent, or tool. It can be used as a hosted gateway or self-hosted, with hosted inference and Pro offered alongside the open-source gateway.

Free Claude Code is an independent open-source local proxy for routing Claude Code, Codex, Pi, OpenCode, and other coding-agent requests to selected free, paid, subscription, or local model providers while preserving their existing APIs. It provides a searchable model catalog and an Admin UI for configuring providers, supports automatic fallback to another configured model after provider retries are exhausted, and can be launched from a terminal, desktop app, IDE, Discord, Telegram, or phone. Optional RTK filtering reduces common terminal-output tokens, while voice input can use local Whisper or NVIDIA NIM transcription. The project states that it is not affiliated with or endorsed by Anthropic, and that provider free-tier availability and limits may change.

vLLM is an open-source library and inference-serving engine for deploying large language models on GPUs and other hardware accelerators. Originally developed in UC Berkeley's Sky Computing Lab, it is maintained by a broad community and provides an OpenAI-compatible API server, as well as Anthropic Messages API and gRPC support. Its serving stack manages attention key-value memory with PagedAttention, continuous batching, chunked prefill, prefix caching, streaming generation, and disaggregated prefill, decode, and encode. It also supports speculative decoding, quantization, optimized attention and GEMM/MoE kernels, automatic kernel generation and graph-level transformations, structured outputs, tool calling, multiple decoding algorithms, and distributed inference through tensor, pipeline, data, expert, and context parallelism. The project integrates with Hugging Face models and supports decoder-only, mixture-of-experts, hybrid state-space, multimodal, embedding, retrieval, reward, and classification models. It can run across NVIDIA, AMD, Intel, and other supported accelerators, as well as x86, ARM, and PowerPC CPUs, and can be installed with uv or pip or built from source.