wtf is pareto?

Pareto is a tool for cleaning up and curating the data robotics teams use to train AI models. Robotics teams often record far more demonstration videos and sensor logs than actually end up in a training set, and deciding which ones are good enough is normally a manual and inconsistent process. Pareto tries to replace that guesswork with measurable checks. It works with datasets in the LeRobot format, a common way of storing robot demonstrations made of synchronized multi camera video, robot states, actions taken, and a description of the task. After you upload a dataset, Pareto measures how consistent the demonstrations are with each other, groups similar demonstrations together to reveal patterns in behavior, flags likely duplicates or gaps in coverage, and lets you search across every recorded episode using plain text, such as searching for a specific object that appears in the footage. Once you have identified which episodes are worth keeping, Pareto can export a new, clean dataset in the same LeRobot format ready for training. This repository is the free, self-hostable community edition of a system that the company Hebbian Robotics also offers as a paid, managed service. The community edition includes the same indexing, search, API, and web interface, and is meant for local testing or use within a single trusted organization rather than serving multiple separate customers from one deployment. Getting started means installing a command line tool built in Rust using Cargo, then pointing it at a local dataset to index it, check consistency, search it, or export a curated subset. Running the full web interface and background processing additionally needs Postgres and a Docker Compose setup provided in the repository. Every command can also output structured JSON, and the tool documents a way for a coding agent to use it directly to investigate a dataset and record findings. This is aimed at robotics engineers and researchers responsible for preparing training data for robot learning models.