wtf is rag-failure-playground?

RAG Failure Playground is an interactive website that teaches people how a common AI technique called RAG can go wrong. RAG stands for retrieval augmented generation, a method where an AI system first searches a collection of documents for relevant information, then uses that information to write an answer, instead of relying only on what the AI already knows. This project walks through thirteen distinct ways that process can fail, each labeled with a code like FP1 through FP13, such as missing content that should have been found, citing sources that do not actually support the claim, or pulling in outdated information from a stale index. The site runs entirely in your web browser, with no backend server and no configuration needed. It uses a real search and ranking pipeline, including a text-matching technique called BM25 and an AI embedding model that runs directly in the browser to understand meaning, so the demonstrations are built on genuine calculations rather than staged results. However, since there is no live AI language model answering questions in the browser, the answers shown for each failure scenario are pre-written by the developer specifically to produce the kind of flawed results being taught, and this is clearly disclosed rather than hidden. For each of twelve curated scenarios, the tool shows a trace of what the pipeline did step by step, a table checking whether each cited source actually supports what it is cited for, and scores measuring things like how much relevant context was recalled and how faithful the answer was to the source material. There is also a ship check mode that runs all twelve scenarios at once and checks whether the results pass or fail against thresholds you can configure, meant to reflect how real production RAG systems are expected to tolerate some failures rather than achieve perfection. This project is aimed at developers and technical learners building AI systems that search documents to generate answers, offering a hands-on way to recognize failure patterns before encountering them in a live product. Running it locally requires Node.js and npm, and it can also be deployed as a static site to a service like Vercel with no extra configuration.