wtf is cameo?

CAMEO is a research project that helps AI image generators produce more consistent results when creating multiple views of the same object or scene. If you have ever seen an AI generate an image of a chair from the front, then try to show the same chair from the side, only to have it look like a completely different chair, this project tackles that exact problem. When AI models generate multiple viewpoints, they use an internal mechanism called "attention" to figure out which parts of a new view should correspond to parts of the reference image. The researchers behind this project discovered that this attention mechanism naturally learns some of these correspondences during training, but it is imperfect and breaks down when the angle between views gets large. This project improves the process by directly training that attention mechanism with known geometric correspondences, teaching it more precisely how points in one view relate to points in another. The main audience is researchers and engineers working on multi-view image generation or 3D asset creation from 2D images. A game studio wanting to automatically generate a full 3D model of a prop from a single photograph, or an e-commerce platform showing a product from every angle based on one product shot, would benefit from the improved view consistency this approach provides. What makes this approach notable is its efficiency. Supervising just a single attention layer is enough to guide the entire model toward learning precise correspondences. This halves the number of training iterations needed to converge while still achieving better results at the same number of steps. The technique is also model-agnostic, meaning it can be applied to essentially any multi-view diffusion model, not just one specific architecture. The code is built on top of an existing project called MVGenMaster, and the dataset format expects sequences of images paired with camera information like intrinsics and extrinsics, ordered from reference frames to target frames.