wtf is x-dub?

X-Dub is a tool that takes any video of a character and replaces their mouth movements to match a new audio track. Imagine dubbing a movie into another language, but instead of the actor's lips being out of sync with the new voice, the video is regenerated so the mouth movements perfectly match the words being spoken. You just provide a video and an audio file, and it outputs a new video where the character appears to be speaking the audio you supplied. Under the hood, it uses an AI model (built on top of a video generation system called Wan) to analyze the original video, detect the face, and then regenerate just the lower portion of the face so the lips sync to your audio. It automatically crops to the face area, does the lip-sync generation at a fixed resolution, and then maps the result back onto the full original video so everything else stays intact. You can tweak settings to balance how closely the result matches the original face versus how strongly the mouth follows the audio. This would be useful for anyone doing video localization, content creators making dubbed videos, or filmmakers who need to fix or replace dialogue without reshooting. It also handles non-human characters like cartoons and animals, which broadens the use cases to animation and game content. The current release only supports single-person videos, so it works best for close-up or solo shots rather than scenes with multiple speakers. A few practical notes: it requires a fairly powerful graphics card (around 21 GB of video memory), so it's not something you can run on a standard laptop. The released version is a public adaptation of the model described in the research paper, the original internal model couldn't be open-sourced, so this public version uses a different underlying architecture with similar results. There are some known limitations, including occasional flickering, minor identity or color drift, and noisy frames in about 2% of cases. It also runs roughly twice as slow as the internal version. The team is actively working on improvements including multi-person support, better cropping, and faster inference.