Serve Quality API

OpenAI API is a cloud service that provides access to OpenAI's AI models, enabling developers to integrate language processing capabilities into applications. It is used for tasks such as generating text, building chatbots, and orchestrating context and model selection. The API is commonly employed as a backend for various AI-powered tools and services.

ai-evaluation-framework is an open-source framework by Dreamers Inc. for benchmarking model-based solutions by accuracy, p95 latency, and estimated cost against user-defined ground-truth cases. Each task specifies fields and a scoring rule per field, including exact identifiers, normalized dates, numeric tolerances, containment, token overlap, and expected-missing values; cases can also provide alternate accepted renderings. It runs models from OpenAI, Anthropic, or an OpenAI-compatible gateway, compares results in a table, emits detailed JSON reports, and caches responses by model, task, and case so scoring rules can be changed without another model call. The repository uses document-field extraction as its worked example but accepts arbitrary text inputs such as OCR output, transcripts, and PDF text. A separate feedback server collects thumbs-up or thumbs-down responses with required comments, stores append-only JSONL data, and exposes statistics, exports, and candidate cases for human review; it does not train models. The repository includes tests using scripted mock models and is licensed under Apache-2.0.

OpenAI Evals is an open-source framework for evaluating large language models and systems built with them, including tool-using agents and prompt chains. It includes a registry of benchmark evals, supports custom model-graded and private evals based on a user's data, and uses a completion-function protocol for advanced workflows. The package can be installed with pip and run locally with an OpenAI API key; the repository also documents optional result logging to Snowflake and configuration through the OpenAI Dashboard.

Lila is the open-source server software that powers lichess.org, developed by Thibault Duplessis (ornicar) and community contributors. The codebase is written in Scala (Play Framework) and runs on the JVM, implements real-time and correspondence chess, tournaments, puzzles, studies, user accounts, forums and a JSON API, and integrates with chess engines via the UCI protocol (commonly Stockfish) for computer play and analysis. The source is published under the GNU AGPLv3 on GitHub.

AI Engineering from Scratch is a free, open-source curriculum developed by rohitg00 for learning to build and ship AI systems end to end. Its 20-phase sequence covers development tooling, mathematical and machine-learning foundations, deep learning, NLP, computer vision, generative AI, LLM applications, agent engineering, Model Context Protocol (MCP), agent skills, reinforcement learning, and related topics, with implementations in Python, TypeScript, Rust, and Julia. The repository describes 511 lessons and approximately 329 hours of material; each lesson produces a reusable artifact such as a prompt, skill, agent, or MCP server. Learners are instructed to read the lesson documentation, type and run the code from the repository root, record command evidence and outputs, and make a small change before continuing. It provides routes for complete foundations, mathematics and machine learning, production LLM applications, agent engineering, MCP, agent skills, and Claude certification preparation. The repository is distributed under the MIT license and is accompanied by a website containing the same lesson content.