Study whether internal model signals predict hallucinations better than output confidence scores.
Reproduce a preregistered machine learning research campaign on a laptop CPU in under two minutes.
Learn about the answer identity confound as a pitfall to check for in similar error-detection research.
Explore public trace and feature data from a 24,500-example hallucination detection campaign on Hugging Face.
| solarkyle/jspace | develp10/rustinterviewquiestions | fukikomarga/exodus-fake-balance | |
|---|---|---|---|
| Stars | 48 | 48 | 48 |
| Language | Python | Python | Python |
| Setup difficulty | easy | moderate | moderate |
| Complexity | 4/5 | 2/5 | 2/5 |
| Audience | researcher | developer | general |
Figures from each repo's GitHub metadata at analysis time.
Reproducing the headline results needs only pip-installed dependencies and a CPU, no GPU required.
jspace is a research project that asks whether you can peek inside a large language model's internal workings to predict when it is about to give a wrong answer, rather than just trusting how confident the model sounds. The model under study is Google's Gemma 4, and the internal signal comes from a technique called the Jacobian lens, a way of reading activity inside the model as it processes a prompt. The README describes a large, carefully planned experiment covering over 25,000 prompts across 13 public test sets and six subject areas. Before looking at the results, the researchers wrote down exactly what would count as success or failure for each stage, a practice called preregistration, meant to keep the analysis honest. They trained a small classifier, a simple model that learns to sort things into categories, on these internal signals and found it predicted wrong answers better than the model's own stated confidence did, and this held up when tested on related but different datasets within the same type of task. However, when the frozen classifier was tested on five completely new, never-seen datasets, the results did not hold up as a universal rule. It kept working on question-answering tasks that involve looking up facts, but broke down on tasks that ask the model to judge whether a statement is true, sometimes even flipping direction. The researchers are upfront that they do not yet fully understand why this happens. A notable side finding is a methodological warning for anyone doing similar research. On test questions where the correct answer never changes, such as always answering yes to a certain type of question, this kind of internal signal probe can appear to detect errors extremely well while actually just reading which answer the model was already leaning toward, not genuinely detecting a mistake. The authors call this an answer identity confound and recommend that other researchers check for it in their own published results. The project includes a script that reproduces the main results on a normal computer in under two minutes using only a CPU, along with public data, reports, and code. The underlying Jacobian lens method credits Anthropic's research and is Apache 2.0 licensed, while the scripts written for this repository are MIT licensed.
A preregistered research project testing whether internal LLM signals can predict wrong answers better than the model's own confidence, with mixed results.
Mainly Python. The stack also includes Python, LightGBM, Gemma.
Scripts in this repository are MIT licensed, and free to use for any purpose, the underlying Jacobian lens method it builds on is Apache 2.0 licensed from Anthropic.
Setup difficulty is rated easy, with roughly 5min to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
Don't trust strangers blindly. Verify against the repo.