wtf is cortex?

Cortex is a research framework for controlling robots that need to complete long, multi-step physical tasks, such as a sequence of chemistry-style lab actions rather than a single quick motion. It also goes by the name InternVLA-M1.5. The core idea is to split the work between two AI models that stay aligned with each other. A higher-level model acts like a planner: it watches the scene, keeps track of what has already happened using a running memory, and breaks the overall goal into smaller subtasks. A lower-level model then handles the physical execution of each subtask, reacting moment to moment to actually move the robot. The project provides three main things. First, a large dataset of over 4,000 hours of annotated long-horizon subtask examples, pulled from multiple existing robot datasets and benchmarks. Second, an automated three-stage workflow for labeling this kind of data: an AI model generates first-pass labels, a person corrects a small sample of these in a browser, and the corrections are then spread across the rest of the data automatically. Third, a ready-to-download planning model that can be used directly for zero-shot long-horizon planning, meaning it can plan for new tasks it was not specifically trained on. The README reports that Cortex outperforms comparison models on internal benchmarks, including general-purpose systems like GPT-5 and Gemini, on tasks involving spatial reasoning, counting, and multi-step logical consistency. It also reports stronger results on simulated benchmarks and, notably, a large jump in success rate on real-world manipulation tasks compared to a prior method. Using Cortex requires following a setup guide before running any evaluation, and the examples in the README involve setting environment variables, downloading a released model checkpoint from Hugging Face, and submitting evaluation jobs through a cluster scheduler called sbatch. There are separate evaluation modes for testing individual steps versus testing a full closed loop across an entire task, plus a visualization script for reviewing subtask predictions. The README also documents how to set up API credentials for the labeling workflow, with a note to keep them in the shell environment rather than in any script or committed file. The full README is longer than what was shown.