AI product Open source
An open-source Python tool from Pathway that generates original ARC-AGI-1-style grid puzzles with a distribution matched to the public evaluation set. It produces fresh tasks for private model evaluation, filters near-duplicates and malformed outputs, and writes a tasks.json file in the standard ARC format with train and test sections. The generated tasks are compatible with existing ARC evaluation harnesses and accompany Pathway's BDH-CQ model evaluation work.
3 uses taken from transcripts — each links to the moment in the video.
A generator for creating ARC-style puzzles whose distribution matches a public evaluation set. It outputs standard ARC JSON and is provided alongside Pathway’s model.
An open-source Python tool from Pathway that generates ARC-AGI-1-style reasoning tasks matching the distribution of public evaluation sets. It creates private, unseen tasks in a standard format for comparing model performance beyond public benchmark familiarity.
Generates fresh grid puzzles matching the measurable characteristics of the public ARC evaluation set without publishing another benchmark. It filters near-duplicates and malformed outputs and emits standard ARC JSON.
3 in the library.