Microsoft Research's Mirage gives video generation a persistent spatial memory that doesn't forget what's around the corner
Most video generation models struggle with spatial continuity - pan the camera away from a scene and return to it, and details have often shifted or disappeared entirely. Mirage, a collaborative project from Microsoft Research and several universities, addresses this by giving the model a persistent memory of the space it has already generated, so previously seen areas remain coherent when revisited.
The core technical distinction in Mirage is where scene information is stored. Traditional approaches often rely on pixel-based point clouds - explicit 3D representations derived from rendered frames. Mirage instead encodes and retains scene data directly in latent space, the compressed internal representation that diffusion-based models already work within. This means the system does not need to reconstruct geometry from pixels every time it needs to reference what came before.
That design choice has practical consequences beyond consistency. Working in latent space rather than maintaining dense point cloud structures cuts both processing time and graphics memory consumption meaningfully, which matters for research scalability and any potential downstream deployment. The result is a model that can handle extended camera trajectories - moving through a corridor, circling a room - without the scene fragmenting or contradicting itself across segments.
The system is not without its current boundaries. Mirage handles static environments well but has not yet solved the harder problem of tracking moving objects reliably across video segments. A person or vehicle that exits the frame and re-enters may not be rendered consistently, which limits the model's usefulness for dynamic scene simulation. That gap points to the next natural area of development for world models of this kind - integrating persistent spatial memory with robust object-level tracking to handle scenes where not everything stays still.
