wtf is bitnet?

BitNet (bitnet.cpp) is Microsoft's official framework for running 1-bit large language models efficiently on ordinary CPUs and GPUs. A standard large language model stores each number in its weights using 16 or 32 bits of precision. BitNet's approach dramatically reduces that to just 1.58 bits per weight, each weight can only be -1, 0, or +1. This radical compression means models take up far less memory and can be computed much faster using simpler math operations, enabling large AI models to run on devices that would normally struggle with them. The framework provides optimized inference kernels, specialized low-level code that performs the math as efficiently as possible, for both ARM processors (common in Apple Silicon and mobile chips) and x86 processors (standard desktop and server CPUs). According to the README, it achieves speedups of roughly 1.4 to 6 times over standard approaches while reducing energy consumption by 55 to 82 percent depending on the hardware. As a practical demonstration, a 100-billion-parameter model can reportedly run on a single consumer CPU at a speed comparable to human reading pace. GPU inference support was added in 2025. You would use BitNet when you want to run a capable language model locally on your laptop or desktop without requiring a powerful GPU, or when building applications for edge devices, embedded systems, or scenarios where energy efficiency matters. It is also relevant for researchers studying efficient AI model design. The project is built in Python and C++, uses CMake for compilation, and requires Clang 18 or newer as the compiler. Pre-built models are available on Hugging Face.