wtf is dexhoi2sim?

DexHOI2Sim is a research tool for testing whether a computer-generated hand grasping an object actually works in a physics simulation, rather than just looking correct in a video. It takes a recorded or generated hand and object interaction and replays it inside physics engines called MuJoCo and IsaacGym, then reports whether the grasp physically holds up and how accurately the object followed its expected path. The project builds a detailed 3D hand model with 45 points of movement based on the MANO hand format used in computer vision research. Instead of approximating the hand's shape, it computes the exact joint angles directly from the recorded hand pose, and the tool has been checked to closely match the original MANO model. Because the hand shape comes from a parameter unique to each person, a different subject produces a differently sized and shaped hand. Running the tool involves one command that builds the hand model, simulates the motion, and produces a results file. That file reports things like how far the object drifted from its intended path and whether the grasp counts as successful, meaning the object stayed close enough to where it was supposed to go. The examples in the README show cases where a replayed grasp looks fine when just following the recorded motion exactly, but fails once real physics, like gravity and collisions, are applied. This gap between looking right and actually working is the core problem the tool is built to catch. Beyond built-in datasets, users can supply their own hand and object data as a single file, including support for two hands interacting with the same object, such as one hand passing something to another. A ready-made example is included so the tool can be tried out without needing to download any external datasets. The project is written in Python and is aimed at researchers working on hand and object interaction generation, robotics, or computer vision, who need a way to check whether their generated grasps are physically realistic rather than just visually convincing. Using it requires installing physics simulation software and, for the built-in datasets, registering with the dataset providers to get access to the required models.