wtf is agentic-testbed?

Agentic Testbed is a framework for testing whether an AI agent is actually ready to be used in production, rather than just checking if its answers sound correct. It looks at the agent's full setup: the model it uses, its prompts, the tools it can call, its permissions, the data it touches, and how it behaves when something goes wrong or when a human needs to step in. The end result of running it is not a single confidence score, but a bundle of evidence that supports a decision to approve the agent, approve it with limits, require fixes, or reject it outright. The project is described as the third piece of a larger system for building AI agents responsibly. Two companion projects handle planning and running the agent's work, while this one focuses specifically on testing the finished agent before it goes live. Each of the three pieces can also be used on its own. Using it involves writing a manifest describing the agent being tested, then running a suite of test scenarios against it using a command line tool included in the project. The included example intentionally uses an unsafe agent so that new users can see what a failing, non-ready result looks like in practice. The testing process walks through several stages: registering and isolating the agent, defining its allowed tasks and risk limits, recording its full configuration, running it through a range of normal and adversarial scenarios, and comparing results across runs before making a final decision. The project checks many layers of an agent's behavior, including whether its tools are called correctly, whether it takes a safe path to complete a task, whether it resists attempts to trick or manipulate it, and whether it asks for human approval at the right moments. The author describes this as an early reference version, not a certified or compliance ready product. It is released under the MIT license.