wtf is simple-llm?

SimpleLLM is a from-scratch engine for running large language models, built in under 1,000 lines of Python. Instead of relying on heavy existing frameworks, it provides a compact, readable starting point for running AI models on high-end NVIDIA hardware. Right now, it specifically supports running OpenAI's gpt-oss-120b model on a single NVIDIA H100 GPU. Under the hood, it implements modern performance techniques like continuous batching (where new requests join the active processing group instead of waiting in line), CUDA graphs, and specialized math operations that squeeze maximum speed out of the hardware. You interact with it by loading a model and submitting prompts, and the engine handles the rest. It processes everything in a background loop so the GPU never sits idle waiting for the next task, and its performance is competitive with major production engines like vLLM at both single-user and 64-user batch sizes. This project is designed for researchers, students, and research labs who want to understand or experiment with how AI inference actually works. A student could use it to trace the entire lifecycle of an AI request from start to finish without getting lost in layers of abstraction. A researcher testing a novel way to speed up AI processing could modify this codebase directly, rather than trying to strip down a massive production framework to understand it. What makes this notable is the tradeoff: it currently only supports one specific model on one specific GPU. The author deliberately chose a complex starting point to prove that building a modern, high-performance inference engine from scratch is viable. The project includes specialized community-contributed processing kernels and plans to add support for running across multiple GPUs and supporting other model architectures in the future.