Mastering AI Observability Workshop

AI-Native Workshop is a repository-based workshop for practicing AI-native software development workflows. Its curriculum covers voice coding, agent sessions, loops and measurable goals, Git worktrees, verification gates and hooks, adversarial review, and scheduled tasks. The repository includes setup skills, exercises, slides, a glossary, a facilitator guide, and an optional local `aie-coach` MCP server with seven tools for check-ins and setup scoring. Its setup workflow runs inside Claude Code and can install and verify Bun, the Codex CLI, and Git; the workshop is organized in four blocks covering voice, loops and goals, gates, and schedules.

Braintrust is an AI observability and evaluation platform for agents and other AI systems. It traces production prompts, responses, tool calls, latency, cost, and quality; supports searching logs, real-time trace inspection, dashboards, and custom views; and runs experiments against versioned datasets with scoring by LLMs, code, or humans. Teams can turn production traces into evaluation datasets, compare prompts and models, investigate recurring failures through natural-language queries, and use the results to identify regressions and improve agents. Braintrust provides native SDKs for Python, TypeScript, Go, Ruby, C#, and other environments, an MCP server for querying logs and running evaluations from an IDE, and Brainstore, its database and query engine for AI data. The platform states that it supports framework-agnostic integrations, SOC 2 Type II certification, GDPR and HIPAA compliance, SSO, role-based access control, and hybrid deployment options.

Braintrust CLI is a command-line interface for the Braintrust AI observability and evaluation platform. It initializes projects, queries trace data with SQL, runs evaluations, views logs, creates evaluation datasets, and supports an agent-improvement workflow that connects production failures to code and scorer changes. In the demonstrated workflow, a coding agent uses the CLI to investigate traces, propose changes, add regression examples, and run evaluations.

Project Rosetta Stone is a workshop and comparison repository for instrumenting the same AI agent with different agent frameworks and observability platforms. Its Wonder Toys application is a chat-to-purchase toy-store assistant with semantic vector search, product browsing, purchasing, order tracking, and cancellation; framework variants are provided without observability, with Arize Phoenix instrumentation, and with Arize AX instrumentation. Readers can compare the instrumentation footprint by diffing each framework's baseline and instrumented directories, then run the applications locally to send traces to Phoenix or AX. The repository also includes shared synthetic requests, evaluation harnesses, voice testing for the OpenAI voice tier, product data, and instructions for adding frameworks and running the end-to-end workflows; it is licensed under MIT.

Copilot Workshops is an open-source collection of guided, hands-on lessons for exploring GitHub Copilot's agentic capabilities across the software development lifecycle. Its content covers the Copilot CLI, VS Code agent mode, the GitHub Copilot app, and the Copilot cloud agent, with Markdown lessons, images, and an Astro + Starlight site that publishes the material; the workshop's Tailspin Toys demo application is maintained in a separate repository. The repository is MIT-licensed.