LiteLLM

FreeLLMAPI is an open-source proxy that aggregates free tiers from multiple LLM providers behind a single OpenAI-compatible `/v1` endpoint. It can also connect custom OpenAI-compatible chat, embedding, image, and audio endpoints, including local servers and remote gateways. Its router selects an available model, fails over when a provider is rate-limited, stores provider keys encrypted, and tracks per-key usage against free-tier limits. The router refreshes its model catalog from a signed feed; free installations receive periodic snapshots, while the hosted premium offering provides the live catalog. The project is intended for personal experimentation and is available as a self-hosted install with a hosted service at freellmapi.co.

vLLM is an open-source library and inference-serving engine for deploying large language models on GPUs and other hardware accelerators. Originally developed in UC Berkeley's Sky Computing Lab, it is maintained by a broad community and provides an OpenAI-compatible API server, as well as Anthropic Messages API and gRPC support. Its serving stack manages attention key-value memory with PagedAttention, continuous batching, chunked prefill, prefix caching, streaming generation, and disaggregated prefill, decode, and encode. It also supports speculative decoding, quantization, optimized attention and GEMM/MoE kernels, automatic kernel generation and graph-level transformations, structured outputs, tool calling, multiple decoding algorithms, and distributed inference through tensor, pipeline, data, expert, and context parallelism. The project integrates with Hugging Face models and supports decoder-only, mixture-of-experts, hybrid state-space, multimodal, embedding, retrieval, reward, and classification models. It can run across NVIDIA, AMD, Intel, and other supported accelerators, as well as x86, ARM, and PowerPC CPUs, and can be installed with uv or pip or built from source.

LLM Space v4 is a local-first desktop workbench for prototyping and developing AI agents. It lets users version prompts, system messages, tools, and model settings; trace each model call and tool run in an agent loop; replay historical runs to debug failures; and evaluate agent performance across runs. It can generate prompts and tools with AI and turn a thread into a runnable LangGraph agent. Threads, project files, and API keys remain on the user's computer, while the project describes the app as cloud-ready for managed agents. The desktop application uses Electrobun with a React, Tailwind CSS, and shadcn/ui interface, and is built as a Bun monorepo around Pi Agent Core.

Vercel AI Gateway is a managed service from Vercel that routes application requests to language-model providers, centralizes API key management, and provides observability, caching, and rate-limiting for LLM usage. It is intended to simplify integration of multiple LLMs and enforce access controls for apps and autonomous agents.

Awesome LLM Apps is an open-source GitHub collection of Python templates for AI agents, agent skills, and retrieval-augmented generation (RAG) applications. Its examples include starter and advanced agents, multi-agent teams, voice and multimodal agents, MCP-based agents, memory-enabled agents, data analysis, web scraping, and other application patterns. The repository provides runnable source projects and step-by-step tutorials. Individual agents can be cloned and run with an API key, while agent skills can be installed with a command such as `npx skills add` and used with coding agents including Claude Code, Codex, and Cursor. The projects support models from providers and ecosystems including Claude, Gemini, GPT, DeepSeek, Llama, and Qwen. The repository describes the collection as free and open source under the Apache-2.0 license.