wtf is arc-task-gen?

arc-task-gen is a tool that creates new puzzle style tasks in the same style as ARC-AGI-1, a well known public benchmark used to test how well AI models reason. The tasks it generates are built to match the statistical patterns of the original public evaluation set, so they can be used to test frontier AI models on problems those models have likely never seen before. The repository was built alongside a research paper about a model called BDH-CQ, which combines learning from examples given in context with a form of iterative reasoning carried out inside a continuous latent space rather than through generating a written chain of thought. BDH-CQ builds on an earlier architecture called Dragon Hatchling, described as a post Transformer recurrent design where small neuron like units pass information to each other through low rank interactions and keep track of context in an evolving internal state. According to the README, a 150 million parameter version of BDH-CQ scored 29.5 percent pass at 2 on the public ARC-AGI-1 evaluation set, while costing about $0.0007 per task to run, which the authors describe as far cheaper per task than a comparison model called GPT-5.6 Luna. The architecture has also been tried in larger pretraining experiments up to 600 billion parameters, and the authors say it scales in a way similar to standard Transformers while keeping its own reasoning approach. Because ARC-AGI-1 is a public benchmark, models could in theory have been exposed to its exact tasks before being tested on them. This repository exists to work around that problem: it produces a private set of newly generated tasks with similar properties, in the same JSON format used by the original benchmark, so results on it can be compared against results on the public set using existing ARC evaluation tools. Instructions for generating tasks are provided in a separate instructions file included in the repository. The results were independently reproduced by outside researchers, including a co-author of the original Transformer architecture.