Explore an AI-generated interactive world in real time on a desktop GPU
Research how action-conditioned world models can avoid scene lock-in
Try the hosted ABot World Studio playground without installing anything
Train or fine-tune world simulation models using the released video dataset
| amap-cvlab/abot-world | shepherd-agents/shepherd | aminblg/simpleenglish | |
|---|---|---|---|
| Stars | 2,295 | 2,290 | 2,280 |
| Language | Python | Python | Python |
| Setup difficulty | hard | moderate | easy |
| Complexity | 5/5 | 4/5 | 1/5 |
| Audience | researcher | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Needs an NVIDIA RTX 5090 desktop GPU, CUDA 12.8, and compiling FlashAttention and SageAttention from source.
ABot-World is a research project that turns a single high end desktop GPU into a real time interactive world simulator. Instead of watching a fixed length AI generated video, you can steer the simulation with your own actions, similar to controlling a character in a video game, and the world keeps generating new scenes as you go rather than looping or ending after a set amount of footage. The system runs at 720p resolution and 16 frames per second on a single NVIDIA RTX 5090 desktop GPU, with about 1.2 seconds of latency and using roughly 19 gigabytes of GPU memory. It responds to user actions in real time, so exploring the generated world feels continuous rather than like watching a passive clip. A training method the authors call LongForcing lets the simulation introduce new scenes and dynamics as it rolls forward, so it does not get stuck repeating the same environment or need the user to manually switch prompts to see something new. The project provides a smaller causal model, inference code, a local demo built with Gradio, and an online playground called ABot World Studio where people can try the system without installing anything. The team has also released technical reports and is making a 500 hour video training dataset with action annotations available. Setting it up locally is aimed at people comfortable with machine learning environments. The documented setup was tested on Ubuntu 22.04 with CUDA 12.8, Python 3.12, and an RTX 5090, and installation involves cloning the repository, creating a conda environment, installing a specific version of PyTorch built for that CUDA version, and building two performance libraries called FlashAttention and SageAttention from source, which can involve compiling code and working around glibc version issues on some systems.
A research system that turns a single desktop GPU into a real-time interactive world simulator you can steer with your own actions.
Mainly Python. The stack also includes Python, PyTorch, CUDA.
Setup difficulty is rated hard, with roughly 1h+ to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
Don't trust strangers blindly. Verify against the repo.