Visualize which experts a Mixture-of-Experts model uses while generating text.
Identify rarely used experts in a model to prepare for pruning.
Shrink a large MoE GGUF model file by physically removing cold experts.
Run a llama.cpp server that captures live chat routing data for later replay.
| loktar00/visual-llm | 3imed-jaberi/cryptography-si-isamm | 3imed-jaberi/koa-isomorphic-router | |
|---|---|---|---|
| Stars | 2 | 2 | 2 |
| Language | JavaScript | JavaScript | JavaScript |
| Last pushed | — | 2021-09-25 | 2021-02-06 |
| Maintenance | — | Dormant | Dormant |
| Setup difficulty | moderate | easy | easy |
| Complexity | 4/5 | 1/5 | 2/5 |
| Audience | developer | researcher | developer |
Figures from each repo's GitHub metadata at analysis time.
The viewer needs no setup, but capturing your own model requires building a C++ tool inside a llama.cpp checkout.
visual-llm is a browser based tool that lets you watch how a Mixture of Experts language model makes decisions while it processes text. In these models, each token passes through a small set of chosen experts per layer, and this project records those routing choices from a real model running on llama.cpp, then replays them as light moving through one of twelve visual styles, including a spider web, a brain shape, a gravity well, a galaxy, and a metro map. Areas that get used often light up, while experts that are rarely visited stay dark, making it easy to see which parts of the model are doing the work. You do not need to install anything to try it. Cloning the repository and opening index.html in a browser is enough, and three sample recordings are included so you can explore the visualizations before connecting a real model. Keyboard controls let you switch between styles, pause or scrub through the replay, show token labels, turn on a heatmap of expert usage, and toggle between recent and cumulative heat. You can also drag and drop your own recording files in JSONL format. Beyond visualization, the project includes a pipeline for actually shrinking a model. A capture tool built on llama.cpp records routing data, which a script then uses to figure out which experts are rarely used across a set of prompts. Those cold experts can be tested by masking them during inference to see how the model behaves without them, and if the results look safe, another script physically removes those experts from the model file itself, producing a smaller version. The README reports one example where a 35 billion parameter model was reduced from 256 to 192 experts, cutting its file size by about 22 percent, with only tiny differences in output. The tool also supports running llama.cpp as a server so it can capture live chat sessions, browse past recordings, and apply pruning through the interface rather than the command line. Everything in this repository, including the viewer and the pruning scripts, is released under the MIT license.
A browser tool that visualizes how a Mixture-of-Experts LLM routes tokens through experts, plus scripts to prune rarely used experts and shrink the model file.
Mainly JavaScript. The stack also includes JavaScript, C++, llama.cpp.
You can use, modify, and distribute this project freely, including for commercial purposes, as long as you keep the copyright notice.
Setup difficulty is rated moderate, with roughly 5min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Don't trust strangers blindly. Verify against the repo.