Reproduce the ReGround training pipeline to teach a vision-language model to re-examine images mid-reasoning.
Use the pretrained ReGround-Qwen2.5-VL-7B model directly from Hugging Face for inference.
Evaluate a vision-language model on benchmarks like HallusionBench using the included scripts.
Generate self-diagnosis training data from your own visual question answering dataset.
| sespoir/reground | 0xtotem/peek-dspy | ajeygore/llmwiki | |
|---|---|---|---|
| Stars | 42 | 42 | 42 |
| Language | Python | Python | Python |
| Setup difficulty | hard | moderate | moderate |
| Complexity | 5/5 | 3/5 | 2/5 |
| Audience | researcher | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Full reproduction of the paper's training used 32 A100-80G GPUs, requires a GPU cluster and multiple training stages.
ReGround is the code release for a research paper accepted to ACM MM 2026, about a problem in vision-language AI models: when these models reason through a problem in several steps, they can gradually lose track of what is actually in the image they are looking at. ReGround trains a model to notice when this is happening and to look at the image again before giving its final answer. The way this works is that during its reasoning, the model can emit a special marker that signals it needs to re-examine the picture. When that marker appears, the original image is shown to the model again before it writes its final answer. The authors note this approach needs no extra visual tools or changes to the model's architecture, just a change in how it is trained to behave. The repository provides everything needed to reproduce this from scratch for a specific vision-language model called Qwen2.5-VL: code to build the training data, to run two stages of training, to run inference, and to evaluate results. Training happens in two stages. The first stage is supervised fine-tuning using existing tools called LLaMA-Factory and DeepSpeed, and the paper's original run used 32 A100 GPUs, a serious amount of computing hardware. The second stage builds on the first and uses a reinforcement learning method called GRPO, implemented through a framework called veRL, where the reward given to the model combines whether it decided correctly to re-examine the image, whether its final answer was accurate, and whether its output followed the expected format. A pretrained version of the final model is available on Hugging Face for people who want to use it directly rather than retrain it themselves. The project also includes a smoke test script for quickly checking that a running model server responds correctly, and an evaluation script for measuring performance against a benchmark called HallusionBench. Automated tests, including code style checks and a secret scanner, are included in the repository. This is a research project intended for people already familiar with training large vision-language models, not a beginner friendly tool, and it requires substantial GPU resources to reproduce the full training pipeline.
ReGround is research code that trains a vision-language model to notice when it is losing track of an image during multi-step reasoning and look at the image again before answering.
Mainly Python. The stack also includes Python, Qwen2.5-VL, veRL.
Setup difficulty is rated hard, with roughly 1day+ to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
Don't trust strangers blindly. Verify against the repo.