wtf is illusion-of-robustness?

This repository holds the code behind a research paper studying how large language models behave when they are given extra text that has nothing to do with the question being asked. The paper is called The Illusion of Robustness, and its central finding is that while a model's overall accuracy on a benchmark barely changes when unrelated context is added, individual answers underneath that average can flip quite a bit. Some questions get answered better with the irrelevant context added, others get answered worse, and which questions are affected differs from model to model. So a stable looking average score can hide real instability at the level of single predictions. The code lets researchers reproduce this effect. It prepends task irrelevant context, by default random pseudo words, to standalone question answering benchmark questions, then measures how each individual answer's correctness changes. Beyond the overall accuracy and average change, it reports two extra measurements: instability, which is the average size of per question change in either direction, and worst tail degradation, which focuses on how badly the worst affected slice of questions performs. The repository is organized into a pipeline for models accessed through an API, a separate pipeline for locally run open weight models that inspects internal model states more directly, and a set of scripts for building the context and target datasets used in the experiments. There is also a test suite. Setup involves installing Python dependencies and providing API credentials in a configuration file, since the pipeline works with any provider supported by the LiteLLM library. The project releases all of its datasets publicly on Hugging Face, organized by experiment, covering several benchmark sources, different types of irrelevant context such as pseudo words, random web pages, and random tokens, and results across many different models. This makes it possible for others to inspect the exact data and rerun or extend the analysis. The README includes example commands for running both the main API based experiments and the local model experiments.