Evaluation pipeline for high-stakes AI agents (trace, curate datasets, expert-reviewed failure modes, CI quality gates)
A systematic AI engineering loop for customer-facing agents in regulated industries: trace every agent step, mask PII before persistence, build ground-truth datasets from production traces and expert curation, convert domain-expert comments into failure-mode categories, score with a blend of LLM-as-judge and deterministic code evaluators, and block any prompt/config change in CI when scores drop below thresholds.
From AI Engineer — Evals for High-Stakes AI Agents — Doneyli De Jesus & Annabell Schäfer, Langfuse at 12:14
Problem: Agents can fail while traditional observability shows green — e.g. a support agent claiming a refund was issued without ever calling the refund tool, discovered only weeks later. Non-determinism makes LLM outputs unreproducible, and humans cannot manually review thousands of production traces.
For: Teams shipping customer-facing AI agents in regulated industries such as financial services and healthcare, where mistakes carry financial, legal and reputational liability.
Products from this video
Claude ClickHouse GitHub Actions Langfuse Langfuse SKILL.md Whisperflow
Examples
- Banking assistant demo built for financial-institution customers: traces with PII masking, guardrail scores for investment-advice intent, CI quality gates on prompt changes — passed the gate with evaluators for source inclusion, source recall, language match, correctness and no unsolicited upsell
Other takes on AI evaluation and benchmarking
- Voice agent benchmarking harness for picking a model stack
- AI technical due diligence service for investors and acquirers evaluating automation claims
- Event-driven adversarial PR review service as a standing agent job
- An independent evaluation company for foundation models and agents
- Per-agent tracking and evaluation tables for ROI reporting
- Developing specialized AI benchmarks for professional workflows