Weco AI

Aiden is Weco's autonomous multi-agent machine-learning research system. It retrieves research papers and GitHub implementations, proposes and audits ideas, runs training experiments, retains code changes that improve a language model's evaluation score, and submits pull requests for promising results in the Parameter Golf competition. Its research loop repeatedly generates proposals and evaluates them against a fixed metric, with experiment branches and scores made inspectable through Weco's dashboard.

AI-Native Workshop is a repository-based workshop for practicing AI-native software development workflows. Its curriculum covers voice coding, agent sessions, loops and measurable goals, Git worktrees, verification gates and hooks, adversarial review, and scheduled tasks. The repository includes setup skills, exercises, slides, a glossary, a facilitator guide, and an optional local `aie-coach` MCP server with seven tools for check-ins and setup scoring. Its setup workflow runs inside Claude Code and can install and verify Bun, the Codex CLI, and Git; the workshop is organized in four blocks covering voice, loops and goals, gates, and schedules.

ai-evaluation-framework is an open-source framework by Dreamers Inc. for benchmarking model-based solutions by accuracy, p95 latency, and estimated cost against user-defined ground-truth cases. Each task specifies fields and a scoring rule per field, including exact identifiers, normalized dates, numeric tolerances, containment, token overlap, and expected-missing values; cases can also provide alternate accepted renderings. It runs models from OpenAI, Anthropic, or an OpenAI-compatible gateway, compares results in a table, emits detailed JSON reports, and caches responses by model, task, and case so scoring rules can be changed without another model call. The repository uses document-field extraction as its worked example but accepts arbitrary text inputs such as OCR output, transcripts, and PDF text. A separate feedback server collects thumbs-up or thumbs-down responses with required comments, stores append-only JSONL data, and exposes statistics, exports, and candidate cases for human review; it does not train models. The repository includes tests using scripted mock models and is licensed under Apache-2.0.

Mini Parameter Golf is a two-hour workshop version of OpenAI's Parameter Golf challenge, maintained by WecoAI. Participants modify `solution.py` to train a small language model that fits within a 4 MB artifact limit and a 60-second training target, then minimize validation bits-per-byte (`val_bpb`) using a fixed evaluation command. The repository supports Weco-powered autoresearch runs with repeated code proposals and evaluations, configurable step counts, timeouts, logs, and dashboard monitoring. It is intended for educational iteration rather than the original challenge's rigorous evaluation, and includes rules, setup instructions, a baseline, and leaderboard submission guidance.

WolfBench is an interactive online leaderboard for evaluating AI-agent configurations on Terminal Bench runs. It compares average, best, and worst scores; consistency, measured by tasks solved in every run; ceiling, measured by tasks solved at least once; token usage; and total cost. Users can filter and reorder results, export charts, share leaderboard state through URLs, and inspect the underlying Weave traces for commands, messages, and failures. Its evaluation workflow launches Terminal Bench tasks through Harbor, filters incomplete or invalid runs in a Marimo dashboard, and turns valid repeated runs into leaderboard results. A configuration records the benchmark, runner, sandbox, agent, model service, and settings, so versions, provider routing, prompts, resource limits, concurrency, retries, and timeouts remain part of the comparison. The project distinguishes infrastructure failures from task failures and uses repeated runs to expose differences between occasional success and consistent behavior.