wtf is esdmax-r9700-fp8?

esdmax-r9700-fp8 is a low level performance add-on for a specific AMD graphics card, the Radeon AI PRO R9700. It provides custom math routines, written for AMD's HIP programming language, that speed up a particular kind of matrix multiplication used inside large AI language models. These routines plug into vLLM, a popular tool for running AI models efficiently, through a small environment variable switch and a paired fork of the vLLM project. The project targets the numeric format called FP8, an eight bit way of storing model weights that saves memory and can run faster than the more common sixteen bit format, as long as the hardware and software are tuned well for it. This repository is entirely about that tuning: getting a graphics card's memory bandwidth used as fully as possible while running an AI model. Most of the README documents a sequence of performance fixes and the reasoning behind them, benchmarked on a real large language model. Each fix targeted a specific inefficiency, such as work being done one thread at a time instead of spread across many, too few parallel work units to keep the card busy, repeated setup work inside a loop that should only happen once, or too many memory read streams competing with each other. Together these changes roughly tripled the speed of decoding text with the model, measured in tokens produced per second, moving performance closer to the theoretical maximum the memory hardware allows. There is also a caution in the README about how to benchmark this kind of code correctly, since the graphics card has a large on chip cache that can make a test look faster than it would be with real, varied data. This project is aimed at engineers who work on the low level performance of AI inference on AMD hardware, not general application developers. It includes build, test, and benchmark commands, but assumes familiarity with GPU programming concepts. No license information is given in the README.