Witchai

WhichAI is a tool for comparing front-end design output produced by different AI models.

Chai is an AI tool described as a successor to Wekinator and as “the ChucK AI.” In the Earshot demo, it runs a small model on 40–100 millisecond slices of audio.

ai-design-check is a zero-dependency design-rule checker and agent skill for interfaces built with AI coding agents. It reads web projects including React, Vue, Svelte, Laravel Blade, and HTML, as well as SwiftUI, Jetpack Compose, Flutter, and React Native code, and reports findings with the file and line where a rule is broken. Its rules cover accessibility, responsive layouts, type and spacing scales, component reuse, writing and localization, visual language, motion, and data presentation. The checker can apply mechanical fixes without an agent, while its installed skills can measure a screen, write a design brief, and have an agent repair the remaining issues on a new branch. Additional commands scan a running page in Chrome, inventory component variations, test narrow layouts and long-language content, generate a design system, and create before-and-after videos. It runs through the npm package on Node 20+, supports CI output and pull-request annotations, and is licensed under MIT.

ai-evaluation-framework is an open-source framework by Dreamers Inc. for benchmarking model-based solutions by accuracy, p95 latency, and estimated cost against user-defined ground-truth cases. Each task specifies fields and a scoring rule per field, including exact identifiers, normalized dates, numeric tolerances, containment, token overlap, and expected-missing values; cases can also provide alternate accepted renderings. It runs models from OpenAI, Anthropic, or an OpenAI-compatible gateway, compares results in a table, emits detailed JSON reports, and caches responses by model, task, and case so scoring rules can be changed without another model call. The repository uses document-field extraction as its worked example but accepts arbitrary text inputs such as OCR output, transcripts, and PDF text. A separate feedback server collects thumbs-up or thumbs-down responses with required comments, stores append-only JSONL data, and exposes statistics, exports, and candidate cases for human review; it does not train models. The repository includes tests using scripted mock models and is licensed under Apache-2.0.

Weco AI is an autoresearch agent platform for running and inspecting optimization experiments. It repeatedly proposes code changes, evaluates them against a fixed command and measurable goal, keeps changes that improve the score, and discards weaker attempts. Its dashboard presents the search as an experiment tree with scores and diffs, allowing users to branch from promising runs and steer an ongoing search toward a different idea. The workshop materials demonstrate the approach with Mini Parameter Golf and a Flappy Bird controller evaluated across multiple conditions, including hidden validation data and limits on editable code.