wtf is guardrails-llm-filter?

guardrails-llm-filter is a proxy server, written in Go, that sits between your application and an AI model provider such as OpenAI or Anthropic. Instead of sending requests straight to the provider, your app points at this proxy, and the only change needed is the base URL setting. When a request passes through, the proxy scans it with close to 265 built in rules and replaces anything sensitive, such as email addresses, API keys, access tokens, IP addresses, and personal identifiers like Russian passport or tax numbers, with placeholders such as <EMAIL_1>. The AI model only ever sees the placeholder text, never the real data. When the model sends back a reply, the proxy swaps the placeholders back for the original values before the response reaches your app, including replies that stream back piece by piece and tool call arguments the model generates. The detection step relies on regular expressions and checksum validation rather than a machine learning model, so it adds only microseconds to each request and does not call out to any external service. If something goes wrong inside the proxy, it is built to let traffic through rather than block it, so a bug in the filter does not take down your app. There is a built in web console reachable on a separate port, with pages for an overview of activity, the list of detection rules, a sandbox for testing text against the live rules, settings, an audit log of what was masked and when, and monitoring. It also exposes Prometheus metrics and ships with a ready made Grafana dashboard. A detect only mode lets you see what the proxy would mask without actually changing any traffic, which can be switched to full enforcement through the API without redeploying. The project supports the OpenAI style chat completions and responses endpoints as well as the Anthropic messages endpoint, covering both plain JSON replies and streaming ones. It ships as a Docker image and can also be built from source with a standard Go toolchain. It is released under the Apache 2.0 license, and a companion project packages the same detection engine as an Envoy sidecar for teams using a service mesh.