BTQL

Braintrust CLI is a command-line interface for the Braintrust AI observability and evaluation platform. It initializes projects, queries trace data with SQL, runs evaluations, views logs, creates evaluation datasets, and supports an agent-improvement workflow that connects production failures to code and scorer changes. In the demonstrated workflow, a coding agent uses the CLI to investigate traces, propose changes, add regression examples, and run evaluations.

Braintrust is an AI observability and evaluation platform for agents and other AI systems. It traces production prompts, responses, tool calls, latency, cost, and quality; supports searching logs, real-time trace inspection, dashboards, and custom views; and runs experiments against versioned datasets with scoring by LLMs, code, or humans. Teams can turn production traces into evaluation datasets, compare prompts and models, investigate recurring failures through natural-language queries, and use the results to identify regressions and improve agents. Braintrust provides native SDKs for Python, TypeScript, Go, Ruby, C#, and other environments, an MCP server for querying logs and running evaluations from an IDE, and Brainstore, its database and query engine for AI data. The platform states that it supports framework-agnostic integrations, SOC 2 Type II certification, GDPR and HIPAA compliance, SSO, role-based access control, and hybrid deployment options.

Loop is Braintrust’s AI agent for investigating and improving production agents. It queries trace data with SQL, examines representative failures, and turns findings into evaluation datasets for regression testing. Loop can also generate prompts, scorers, and datasets for optimization, as well as task-specific annotation interfaces for reviewing trace data.

BAML is a language and runtime for defining structured model interfaces and connecting agents to application code. It can repair malformed JSON outputs and was used with GPT OSS120B to find comments mentioning sponsors.

DBX is an open-source, cross-platform database client developed in the t8y2/dbx project. It provides desktop, web, Docker, and CLI distributions for connecting to more than 100 databases and related systems, including MySQL, PostgreSQL, SQLite, Redis, MongoDB, DuckDB, SQL Server, message queues, and service registries. The desktop application uses Tauri, Vue, TypeScript, and Rust and is distributed as a small native binary without a bundled Java runtime, Python environment, or Chromium. Its query editor uses CodeMirror 6 with SQL completion, formatting, diagnostics, history, saved snippets, and file execution. Schema and data tools include object browsing, table editing, ER diagrams, schema comparison, explain plans, field lineage, imports, database transfers, exports, data comparison, and previews of Parquet, CSV, and JSON files through DuckDB. Specialized interfaces support Redis and MongoDB, while consoles cover Kafka, RocketMQ, RabbitMQ, Pulsar, MQTT, Nacos, Consul, ZooKeeper, and etcd. DBX includes an AI SQL assistant that converts natural-language requests into SQL, explains and optimizes queries, fixes errors, and checks generated SQL before execution. It supports Claude, OpenAI, local Ollama models, and OpenAI-compatible endpoints. A separately distributed Rust MCP server lets compatible coding agents such as Claude Code, Cursor, and Windsurf list connections, browse tables, execute SQL, and open tables in DBX; access can be restricted to read-only, safe-write, or high-risk-write modes. The project also provides a CLI for terminal and agent workflows, a signed and sandboxed plugin system with Go and TypeScript SDKs, encrypted credential storage, SSH tunnels, and Docker-based self-hosting. It is licensed under Apache-2.0.