wtf is minivlmdoceval?

MiniVLMDocEval is a research project that tests whether small vision language models, meaning AI models that can read both images and text but have under one billion parameters, are good enough for document understanding tasks. The author evaluated six open source models under one billion parameters against five different document benchmarks, then analyzed where the best model still struggles and tried a targeted fix. Across benchmarks covering chart reading, document question answering, information extraction from images, text recognition accuracy, and table based question answering, a model called Qwen3.5, at 0.8 billion parameters, came out on top overall. Interestingly, the smallest model tested, at 0.45 billion parameters, still beat two larger 0.9 billion parameter models and won the chart reading benchmark outright, suggesting that how a model processes image resolution matters more than raw parameter count. Across all six models, the weakest area by far was visual table lookup, especially in a Wikipedia style table format, while the same models did comparatively well on financial tables. The author then tried to close that table reading gap with a targeted fine tuning approach called LoRA, but the attempt failed a pre registered check meant to catch models that get worse elsewhere while improving on one task: after fine tuning, table performance dropped sharply and document question answering performance also fell. The report treats this as a useful finding in itself, tracing the failure to a mismatch between how the model was trained to answer and how it was scored, and it proposes a corrected approach. The project is built on top of an existing evaluation toolkit called VLMEvalKit, pinned to a specific version, and everything is designed to be reproduced exactly, using fixed random seeds and a fixed sample size. Reproducing the full set of experiments requires a CUDA capable graphics card and works either through a provided Colab notebook or a set of command line scripts. The repository does not state a license.