We Must Pace the Frontier

Anthropic is an AI research and deployment company founded by former OpenAI researchers (including Dario Amodei). It develops large language models such as Claude and focuses on AI safety research and commercial model deployments.

ai-evaluation-framework is an open-source framework by Dreamers Inc. for benchmarking model-based solutions by accuracy, p95 latency, and estimated cost against user-defined ground-truth cases. Each task specifies fields and a scoring rule per field, including exact identifiers, normalized dates, numeric tolerances, containment, token overlap, and expected-missing values; cases can also provide alternate accepted renderings. It runs models from OpenAI, Anthropic, or an OpenAI-compatible gateway, compares results in a table, emits detailed JSON reports, and caches responses by model, task, and case so scoring rules can be changed without another model call. The repository uses document-field extraction as its worked example but accepts arbitrary text inputs such as OCR output, transcripts, and PDF text. A separate feedback server collects thumbs-up or thumbs-down responses with required comments, stores append-only JSONL data, and exposes statistics, exports, and candidate cases for human review; it does not train models. The repository includes tests using scripted mock models and is licensed under Apache-2.0.

DeepSeek and Destroy (DSD) is a long-horizon coding-agent skill for executing large implementation plans through persistent worker loops. A premium parent agent handles architecture, decomposition, decisions, and phase gates, while fresh lower-cost specialist workers implement, review, repair, and recheck repository-scale changes. The default worker backend is external OpenCode using opencode-go/deepseek-v4-flash, and the skill is designed for Codex, Claude Code, OpenCode, Kilo Code, and comparable coding harnesses. Each task is rendered into an immutable JSON contract and managed through attempt directories containing prompts, reports, logs, state, and scope baselines. Its normal loop uses an Implementer, a fresh independent Reviewer, and—when needed—a fresh Fixer followed by another Reviewer. Python-based gates check objective integrity such as hashes, lifecycle, and exact source movement; an optional read-only Evidence Clerk interprets already-produced evidence for the parent but cannot invent proof, rerun verification, repair code, waive integrity failures, approve work, or recurse into another Clerk. Documentation is split between always-relevant parent rules and cold-loaded workspace, harness, transport, compaction, prompt, and specialist-role materials.

In Parallel is a platform that turns the conversations, documents and decisions around an initiative into persistent business context that teams and their AI tools and agents can use. It is positioned to provide living context for collaboration and for AI-driven workflows.

FrontierHarness Eval is an open evaluation benchmark and repository for measuring how coding-agent harness configurations affect software-engineering task results when the underlying model and runtime are held constant. It publishes benchmark definitions, task instructions and metadata, harness-version metadata, normalized aggregate and task-level results, and an agent-neutral workflow for evaluating additional harnesses. The benchmark runs the same model through multiple harness configurations on 30 tasks, using a frozen golden checkpoint and a fresh restore for each task. Its scripts provision the environment, run trials, record verifier-based pass/fail outcomes, measure cost, cache behavior, and speed, then normalize results and generate charts and reports. The repository also provides a CLI and skill-based workflow for evaluating a third-party harness, with infrastructure failures marked separately from task failures. The project focuses on terminal-based software-engineering tasks and states that its results may not generalize to other knowledge-work domains. The repository is published under the frontier-harness-eval organization, with isolated runtimes and checkpoint restores provided by Runta.