LLM Guard

rLLM is an open-source framework from UC Berkeley's Sky Computing Lab for training language agents with reinforcement learning. It connects agent code to evaluation and training through a decorator-based Python API or CLI workflow: agents run in sandboxes, the model gateway captures token IDs and log probabilities, traces are organized into episodes and trajectories, evaluators compute rewards, and a training backend updates the model. It supports existing or custom agent harnesses, Docker, Daytona, Modal, and local sandboxes, with training backends including verl, tinker, and Fireworks, and methods including GRPO, REINFORCE, RLOO, supervised fine-tuning, and on-policy distillation. The project also provides integrated benchmarks and requires Python 3.11 or later.

LiteLLM is an open-source AI gateway and Python SDK from BerriAI for calling more than 100 LLM providers through a unified OpenAI-compatible interface. It supports direct library integration and a self-hosted proxy server, with provider-specific formats translated into common endpoints such as chat completions, responses, embeddings, image generation, audio, batches, reranking, messages, and agent-to-agent calls. The proxy provides centralized authentication and authorization, virtual keys, per-project and per-user spend tracking, logging, guardrails, caching, load balancing, routing with retry and fallback logic, and an administration dashboard. LiteLLM also exposes MCP tools and A2A agents through the gateway and can be deployed as a componentized stack with separate gateway, backend, and UI services. The repository describes it as open source and documents Docker, local development, and Terraform deployment options for AWS and Google Cloud.

GuideLLM is an open-source platform for benchmarking and evaluating real-world LLM inference deployments. It simulates end-to-end interactions with OpenAI-compatible and vLLM-native servers, using real or synthetic text and multimodal datasets and configurable execution profiles such as synchronous, concurrent, throughput, constant-rate, Poisson, and sweep workloads. It measures SLO-oriented latency and token statistics, including time to first token, inter-token latency, and end-to-end behavior, then produces console, JSON, CSV, or HTML reports for deployment tuning, capacity planning, and regression tracking. The tool provides both a CLI and API, with multiprocessing, threading, asynchronous execution, and Hugging Face, file, synthetic, or custom data sources.

AgentGuard is a web-based scanner for AI-agent Skills, including SKILL.md instructions. It combines deterministic pattern checks with AI-assisted analysis to identify capabilities and risks such as shell-command execution, file-system access, environment-variable and credential access, network requests, external downloads, suspicious URLs, obfuscated instructions, prompt-injection patterns, and destructive operations. It produces a trust report with security and quality scores, a risk rating, detected capabilities, findings, explanations, and recommendations. The scanner is free to use, requires no account, accepts up to 80,000 characters of Skill text, and is designed for analysis before installation.

Runlayer is an AI control and enablement platform for companies deploying agents across teams and AI clients. It provides a governed MCP gateway, a catalog of MCPs, skills, and plugins, identity and permission controls, agent-session visibility, spend monitoring, and runtime security checks for tool calls, outputs, intent, and sensitive data. Runlayer distributes approved tools and reusable agent capabilities across clients, packages internal APIs, workflows, and company knowledge, and can create governed AI coworkers with scoped MCPs, skills, memory, triggers, and permissions. The company also describes distilling reusable skills from successful and failed agent traces so that hard-won procedures can be shared across agents; the video presents this as a way to improve task performance while guarding against risks such as prompt injection, PII exposure, unmanaged agents, and unapproved MCPs.