wtf is lacuna?

LACUNA is a research testbed for studying machine unlearning, which is the problem of trying to make an AI language model forget specific pieces of information it learned during training. What makes LACUNA different is that it starts by deliberately planting made-up personal information, like fake names and email addresses, into known and exact locations inside the internal numbers of a language model, so researchers know precisely where each fact lives before they try to remove it. This lets a researcher measure something that is normally impossible to check: whether an unlearning method actually erases a fact from the specific internal parameters that stored it, or whether it merely stops the model from saying that fact out loud while the information is technically still there and can resurface later. The project calls this second, weaker outcome a resurfacing risk, since a stronger prompting method might bring the hidden fact back out. This particular repository is the evaluation release, meaning it assumes you already have one of the provided models, built on the OLMo architecture at either 1 billion or 7 billion parameters, and it walks through the steps of applying your own or an included unlearning method to it, then scoring the result. Scoring happens along three angles: whether the target information is gone while other knowledge stays intact, whether the actual correct internal weights were changed, and how much of the forgotten information comes back if the model is trained a little further on related data. Running it requires a capable GPU, a specific version of Python, and downloading large model files that can take up many tens of gigabytes of disk space, so this is squarely a tool for machine learning researchers working on model safety and privacy, not something a casual user would install. A companion branch of this same repository contains the full pipeline used to build these models from scratch, while this repository focuses only on evaluating unlearning methods against them.