wtf is mcfish?

McFish is a modified version of Fish Speech, a text-to-speech system originally built by Fish Audio. The original software was designed to run on NVIDIA graphics cards, and it did not work well or at all on Mac computers. McFish adapts the system so it runs efficiently on Apple Silicon Macs, turning a tool that previously produced audio at roughly one twentieth of playback speed into one that can generate speech faster than real time. The core problem was speed. The original system ran at about 1.2 tokens per second on a Mac, but the model needs about 21.5 tokens per second to match real audio playback. That meant generating a sentence took roughly twelve seconds before any sound came out, and the entire utterance had to finish processing first. McFish brings generation speed up to between 12 and 23 tokens per second depending on the mode, and it streams audio in chunks so you hear the first sound after about 1.5 seconds instead of waiting for the full clip. The technical changes fall into three areas. First, memory bandwidth improvements: the software sizes its internal caches to the actual request rather than a fixed maximum, skips redundant computations, and offers an optional int8 mode using custom Metal code that reads half the data per token. Second, overhead reductions: the authors fused operations, moved certain checks from every token to every fourth token, and applied PyTorch compilation despite lingering bugs. Third, structural changes allow reusing cached data across text chunks and decoding audio in overlapping segments for faster streaming. Most changes produce bit-identical output, with minor deviations behind flags and validated by listening tests. To use McFish you need an Apple Silicon Mac and a tool called uv. You clone the repository, run a sync, download the model weights (about 11 GB), and then run a Python script with your text to generate a WAV file. You can also provide a 10 to 30 second reference clip to clone a specific voice. For maximum speed, you quantize the model once and run with a compile flag to hit roughly real-time generation. A web UI and API server are included. The project is for research and non-commercial use only, commercial use requires a separate license from Fish Audio. The full README is longer than what was shown.