wtf is embedcache?

embedcache is a small standalone program that sits between your application and any service that turns text into embeddings, the numeric vectors used by AI search and retrieval systems. Services like vLLM, Ollama, text embeddings inference, or OpenAI's own API charge for every embedding they compute, and in practice a large share of those requests are duplicates: the same document re-processed on a nightly job, the same question asked over and over, or several parts of a system embedding the same text without knowing it. embedcache sits in front of those backends and catches the repeat work before it reaches them. It works in a few ways at once. It keeps an exact match cache so identical requests are answered instantly from memory instead of being recomputed. If many identical requests arrive at the same moment, it combines them into a single call to the backend instead of sending them all. When a big batch of text comes in, it only sends the pieces it has not seen before. It can also break a document into chunks and only re embed the parts that actually changed after an edit, rather than the whole document. On top of this it enforces per user token budgets so a runaway process cannot burn through the backend's capacity, and it reports, in plain numbers, how much duplicate spend it caught. It is written in Go with no external dependencies, compiles to one binary, and needs no Python runtime or separate cache server like Redis to operate. You run it once, point your applications at it the way you would point them at a proxy, and it works the same regardless of what language those applications are written in. It supports several embedding providers out of the box, including Ollama, vLLM, OpenAI, Gemini, and others. The project is aimed at teams running their own embedding infrastructure at meaningful scale, or teams building retrieval and AI agent systems that repeatedly re embed the same content. The README documents real measured results from its own test suite rather than only projected numbers.