wtf is millwright?

Millwright is a self-hosted router that sits between your AI applications and the model providers you choose. It accepts requests written in the OpenAI Chat Completions format or the Anthropic Messages format, then forwards each one to an OpenAI-compatible API, Anthropic directly, or Amazon Bedrock, depending on your rules. Instead of every request going to the priciest model, you group models into three roles: cheap, mid, and frontier. Millwright picks the lowest-cost healthy option that your policy allows for that role, based on the type of task or risk level of the request. High-risk or planning-heavy requests are always sent to the frontier role, while routine work can use cheaper models. A notable feature is cache-aware routing. Many AI providers offer discounted pricing when a session keeps reusing the same warm prompt cache, but switching models mid-session can break that savings. Millwright keeps a session pinned to the same confirmed provider and model within each role, so cheap, mid, and frontier work can all run at once without stepping on each other's cache. Every routing decision is logged with the model chosen, the provider, the reason, and any alternatives that were rejected, so you can see exactly where your money is going. A command-line tool shows total spend, cost breakdowns by team, and live activity, and there is a built-in analyzer that can turn usage logs into an HTML or Markdown cost report. Millwright ships as a single Rust binary with Docker support, and your provider API keys stay on your own infrastructure rather than passing through a third-party service. It is built for developers and teams running AI agent tools like Claude Code or similar coding assistants who want to control spend without changing their application code. Setup involves an interactive command that walks through provider endpoints, model roles, and pricing, followed by starting the server.