wtf is orbis2?

Orbis2 is the official code release for a research paper from the University of Freiburg that presents a driving world model, an AI system trained to generate realistic future video of what a car's camera would see next, based on past video frames it has already seen. It is designed for long stretches of predicted driving footage, not just a few frames ahead. The model works in two layers. A first component looks at video sampled at a low frame rate to build a rough, long range sense of what is likely to happen further into the future. A second component then uses that rough context together with recent high frame rate frames to generate the next frames one at a time, filling in fine detail as it goes. On top of this, the model can be steered: you can tell it to follow a specific driving speed and turning rate, or hand it a two dimensional path and have it generate video that follows that trajectory. Using it requires setting up a Python environment through Conda, then downloading pretrained model weights and configuration files from a separate Hugging Face repository. Once those pieces are in place, a single script rolls the model forward from an existing input video, optionally guided by a trajectory file that steers where the generated future frames go. A faster, distilled version of one of the two model components is available for quicker inference at some cost to quality, and several command line options control things like how many future steps to generate, which device handles decoding, and whether to compile the model for speed. This is a research artifact meant to accompany a published paper rather than a polished consumer tool: it assumes familiarity with running Python machine learning code, GPUs, and command line tools. It requires a GPU to run at a reasonable speed. All of the code in the repository is released under the MIT license, so it can be freely reused, modified, and built upon, including commercially.