wtf is pulsar?

Pulsar is a piece of software that lets very large AI language models run on ordinary consumer computers that normally would not have enough graphics card memory to hold them. It does this by keeping the parts of the model that make decisions loaded in the graphics card's fast memory, while storing the much larger set of specialized sub-models, sometimes called experts, on a regular solid state drive and streaming them in as needed, one piece at a time. It is written in Rust and CUDA and is a full rebuild of an earlier project called NeutronStar. The project currently supports nine different AI model designs, including some of the largest publicly available open models, with sizes ranging from tens of billions up to roughly a trillion parameters. On a reference machine with two consumer graphics cards worth 16 gigabytes each, a regular gaming CPU, and one fast NVMe drive, the project reports running these enormous models at speeds from a few tokens per second up to around 50, depending on the model's size and design. Beyond simple support, the README documents a long series of performance improvements and fixes: better handling of long conversations and long documents, a mode that lets the regular computer processor help calculate parts of the model instead of only relying on the graphics card, and corrections to bugs that were quietly making some of the math slightly wrong before. Each improvement is described with before and after numbers from the author's own testing setup. For someone without a technical background, the practical meaning is this: normally, running the very largest AI models requires expensive professional hardware costing many thousands of dollars. Pulsar is an attempt to make that possible on hobbyist grade hardware instead, trading raw speed for accessibility, aimed at developers and AI enthusiasts who want to experiment with cutting edge open models without buying a data center.