wtf is flow-world-models?

FlowWM is a research project from Meta's FAIR lab and Inria that works on teaching a computer to predict what happens next in a video. Existing approaches to this problem force a tradeoff. Some methods can imagine several different possible futures for the same starting point, but they work with a weaker internal representation of the scene. Other methods use a stronger representation of what is in each frame, but they can only predict one single, blurry average outcome instead of distinct possibilities. FlowWM tries to get both benefits at once by predicting multiple possible futures while still using a strong internal representation of the video called DINOv3 features. Alongside the model, the project introduces two benchmarks for testing how well a system predicts the future. The first, called FuturePerception, uses real self driving car footage from the Waymo Open Dataset. Instead of judging predictions by how close individual pixels look, it checks whether an object detector and a depth estimator still work correctly on the predicted future frames, which is a more meaningful test of whether the prediction actually makes sense. The second benchmark, called Bouncing Shapes, is a simple synthetic scene where a circle and a square bounce around a box in ways that could reasonably branch into more than one outcome, letting researchers measure accuracy and variety precisely since every correct answer is known in advance. The repository includes setup instructions using a tool called uv for installing dependencies, scripts to download and convert the Waymo dataset into the format the model expects, and a generator for creating the Bouncing Shapes dataset locally with configurable settings like clip length and resolution. A companion research paper describes the technical design choices in detail for readers who want to go deeper into how the method works.