tonyd2wild/2wild-vision-mode — explained in plain English
Analysis updated 2026-05-18
Give a text-only local language model a way to describe images by piping in captions from a small vision model.
Run live rolling captions from a webcam feed for a local AI setup.
Caption a video file frame by frame for downstream use by another model.
| tonyd2wild/2wild-vision-mode | 0311119/free_registertool | 18597990650-lab/multi-agent-game | |
|---|---|---|---|
| Stars | 24 | 24 | 24 |
| Language | Python | Python | Python |
| Setup difficulty | moderate | hard | moderate |
| Complexity | 3/5 | 4/5 | 3/5 |
| Audience | developer | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Needs a compatible machine (Apple Silicon, NVIDIA GPU, or DGX Spark) and downloads a vision model on first run.
This project, called llm-eyes, gives sight to AI language models that otherwise cannot see anything. Many text-only models, such as DeepSeek V4 Flash and various Llama or Qwen variants, have no way to understand images at all. llm-eyes solves this by running a small, separate vision model on your own computer as a local service. That service looks at an image, a webcam feed, or a video, describes what it sees in plain text, and that description can then be fed into your regular text-only model, effectively letting it "see" through a written caption. The tool works the same way across different kinds of hardware. It supports Apple Silicon Macs, Windows or Linux machines with an NVIDIA graphics card, and NVIDIA's DGX Spark hardware, automatically choosing the right backend for each. A single setup script detects your machine, downloads a small vision model, by default a model called Qwen3.5-0.8B that needs only about 1 gigabyte of storage and 2 to 3 gigabytes of memory, and starts a local server that other programs can talk to. Once running, you can caption a single image, get a running stream of captions from a webcam, or caption a video frame by frame, all from simple command line scripts. The idea is that this small vision model can sit alongside a much bigger text-only model on the same machine, and by piping its captions into that bigger model's prompt, the bigger model gains a working sense of sight without needing to be a vision model itself. The README also describes a setup path meant for AI coding assistants: pointing an agent like Claude Code or Cursor at the repository and asking it to read a specific setup guide file, so the agent can inspect the hardware, install the right pieces, and confirm everything works without the user doing it by hand. The project is released under the MIT license and credits several open source projects, including Qwen, Unsloth, llama.cpp, and MLX, for the underlying models and tools it builds on.
llm-eyes runs a small local vision model as a service that describes images, webcam frames, or video, so text-only AI models can effectively gain sight.
Mainly Python. The stack also includes Python, MLX, llama.cpp.
MIT license: use, modify, and distribute freely, including for commercial purposes, as long as the copyright notice is kept.
Setup difficulty is rated moderate, with roughly 30min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Don't trust strangers blindly. Verify against the repo.