Microsoft Foundry

Azure AI Evaluation SDK is an evaluation framework for Microsoft Foundry users. It is used to test model behavior against application-specific scenarios rather than relying only on general benchmarks, including string-check evaluations and LLM-as-a-judge assessments.

OpenAI is an AI model provider whose language models are available through an API and can serve as one provider in multi-model routing architectures. The videos describe its models being used for AI replies and extraction, Amazon-review analysis and customer-service email generation, legal workflows, complex software development and debugging, and experimental or auxiliary agent tasks.

TrueForge is an open-source, vendor-neutral agent harness developed by TrueFoundry. It runs the agent execution loop, including model calls, MCP tool use, skills, sandboxed code and file execution, approvals, context management, and session state. Agents can use configured model providers, remote MCP servers, git-backed SKILL.md instruction packs, and on-demand sandboxes; context features include subagents, deferred tool loading, Code Mode, large-result offloading, and compaction. TrueForge exposes agents through a bundled chat UI, an HTTP API with a TypeScript SDK, and an embeddable UI SDK. It supports local mode with SQLite and hosted deployments using Postgres and Redis through Docker Compose or Helm; the repository warns that local mode is intended for personal use on localhost rather than production or internet-facing deployment.

Agents That Own Their Inference is a workshop resource for building and operating production AI agents on dedicated GPUs. Its Jupyter notebooks, Kubernetes manifests, load-testing utilities, evaluation code, and Terraform setup guide participants through running a vLLM OpenAI-compatible endpoint and measuring inference behavior, including prefill and decode, time to first token, token generation time, KV-cache capacity, batching, queueing, and saturation. The workshop establishes a baseline, changes serving configurations, reruns realistic load tests, and evaluates output quality alongside throughput and latency. Exercises cover BF16-to-FP8 quantization, speculative decoding with a draft model, vLLM serving-policy changes, dense versus mixture-of-experts behavior, and deploying an agent on the measured inference endpoint. It can use an Akamai-hosted workshop namespace or a participant's Kubernetes cluster with an NVIDIA GPU; the repository is licensed under Apache-2.0.

Fireworks AI is a training and inference platform for transforming open models into specialized, deployable AI systems. Its training offerings range from guided runs, where the platform helps define the task and plan, through configuration-led managed training, to user-written training logic with custom losses, trainers, and reinforcement-learning loops; trained checkpoints can be deployed to production. Its inference service supports open models and customer-trained versions through serverless, dedicated on-demand, and reserved deployments, with an optimized inference engine and OpenAI- and Anthropic-compatible interfaces. Fireworks also provides a model library for running open-source models and supports the continuous train-and-serve workflow described by the videos, including monitoring, hot-loading new weights without downtime, and matching training and serving numerics.