Run the Kimi K3 2.8 trillion parameter model on a CPU-only Linux server without a GPU.
Experiment with mixture-of-experts inference on hardware that cannot fit the full checkpoint in memory.
Prototype local, offline access to a very large language model for testing or research.
Evaluate whether CPU-only inference is viable before investing in GPU infrastructure.
| onetoken-oss/k3flight | karpathy/recurrentjs | codingdocs/typora-markdown-resume | |
|---|---|---|---|
| Stars | 1,015 | 984 | 907 |
| Language | HTML | HTML | HTML |
| Last pushed | — | 2016-11-04 | — |
| Maintenance | — | Dormant | — |
| Setup difficulty | hard | moderate | easy |
| Complexity | 5/5 | 4/5 | 1/5 |
| Audience | developer | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Needs a 64-bit Linux machine with 64GB+ RAM, 1TB+ free storage, and the 929GB model checkpoint downloaded separately.
K3 Flight is a project that lets someone run a very large language model, Kimi K3, on an ordinary computer using only the CPU, with no graphics card required. The full model checkpoint is 929 gigabytes, far more than a typical machine's memory can hold, but the project's runtime, called cPilot Runtime, only needs to keep about 55 gigabytes of active data in memory at any one time. It manages moving the right pieces of the model between storage and memory as they are needed, while the rest of the checkpoint stays on disk. This works because Kimi K3 is what is known as a mixture of experts model, meaning each word it generates only uses a small fraction of its total parameters, 16 out of 896 expert sections. cPilot Runtime takes advantage of that by treating storage, memory, and the CPU as one coordinated system rather than assuming everything has to be loaded up front. The project describes itself as still in preview. The maintainers have completed an internal reference run and are preparing the first public release, including a downloadable Linux binary, checksums, and instructions for getting the model files from Hugging Face. Requirements for that first release are specific: a 64 bit Linux machine, ideally 64 gigabytes or more of memory, at least a terabyte of free storage with a fast local drive recommended, and no graphics card needed. The measured speed in their testing was slow, under one word generated per second, which the authors are upfront about. They frame the project as proof that a model this large can run locally at all, rather than a claim of fast performance. Support for Mac computers is planned for later.
A Linux tool that runs the 2.8 trillion parameter Kimi K3 model on ordinary CPU hardware by keeping only about 55GB of it in memory at once.
Mainly HTML. The stack also includes Linux, GGUF, CPU Inference.
Setup difficulty is rated hard, with roughly 1day+ to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Don't trust strangers blindly. Verify against the repo.