AI product Open source

Langfuse

Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications. It ingests traces for LLM calls and related operations such as retrieval, embeddings, and agent actions, with integrations including OpenTelemetry, LangChain, the OpenAI SDK, LiteLLM, and other LLM frameworks. Its features include prompt management with version control and collaborative iteration, LLM-as-a-judge and code-based evaluations, user feedback and manual labeling, datasets for test sets and benchmarks, trace replay and debugging, metrics, and a playground for testing prompts and model configurations. Langfuse exposes APIs and typed Python and JavaScript/TypeScript SDKs for custom LLMOps workflows. It is available as a managed cloud service or for self-hosting with Docker Compose, virtual machines, Kubernetes via Helm, or Terraform templates.

View repository Visit site Mentioned in 7 videos ↓

What Langfuse is used for

7 uses taken from transcripts — each links to the moment in the video.

  • An LLM observability platform for tracing model calls, retrieval steps, and agent actions. It also manages prompts, runs evaluations, replays traces, and ingests OpenTelemetry data from common LLM frameworks and SDKs.

  • An open-source platform for tracing and evaluating AI agents, combining online observability and monitoring of production traces with offline datasets, experiments and evals. The talk demos its agent skills, prompt management, and annotation features being used by a coding agent to automatically find failure patterns, update datasets and eval criteria, and back-test an improved changelog-writer agent.

  • Open-source agent observability run in the cluster for private tracing of every step, span and tool call

  • Open-source LLM observability platform: tracing, monitoring, evaluators, datasets, experiments; workshop materials at langfuse.com/workshop

  • An open-source platform for tracing, monitoring, and improving AI applications. It captures full agent traces (with PII masking), aggregates metrics and quality scores, hosts datasets and annotation queues for domain-expert review, defines LLM-as-judge and code-based evaluators, runs experiments against datasets, and can trigger CI jobs on prompt changes that act as pass/fail quality gates before agents go to production.

  • Observability provider with built-in evals, mentioned in Q&A

  • An agent observability and LLM ops platform, largely open source, now part of ClickHouse. In this talk it is used to run evaluation workflows for AI agents: building golden data sets, capturing traces of agent runs, scoring them across dimensions with deterministic code, path checks and LLM-as-judge evaluators, and gating promotion to production against thresholds set by domain experts.

Videos mentioning Langfuse

7 in the library.