wtf is needle?

Needle 2 is a very small AI model, only 14 megabytes, built specifically for calling tools and pulling structured data out of text on devices that cannot run typical large language models, like phones, smartwatches, smart home gadgets, and robots. Despite having only 45 million parameters, far fewer than most modern language models, the project reports it performs competitively against other small models many times its size on tool-calling benchmarks, while using about 28 megabytes of memory for a full session. The main way to use it is as a Python package, installed with a simple pip command. A developer describes the functions they want the AI to be able to call, using Python type hints and docstrings the same way many other tool-calling libraries work, and Needle decides which function to call and with what arguments based on a user's request. It can also extract structured data from plain text into a defined shape, such as pulling a vendor name, dollar total, and due date out of an invoice description. Several features aim to make Needle reliable on constrained hardware: it constrains every response to a strict format so it literally cannot produce an invalid tool call, it reports a confidence score with every answer so a program can automatically escalate uncertain cases to a bigger model or a human, and it can automatically pick just the most relevant tools out of a large catalog rather than considering all of them every time, which keeps memory use bounded even in long conversations. The project also includes a browser-based playground for trying out any model without writing code, useful for quickly testing how it responds to different tools and prompts. The underlying model weights are hosted on Hugging Face, and the inference engine downloads and caches itself automatically the first time it runs. Needle is built around a custom architecture the developers call a Simple Attention Network, described in more detail in an accompanying research paper.