Microsoft Research published a proof-of-concept of a project designed to push the boundary of living room immersive entertainment by blending our virtual and physical worlds with projected visualizations.

IllumiRoom uses a Kinect for Windows camera and a projector to blur the lines between on-screen content and the environment we live in allowing us to combine our virtual and physical worlds. For example, our system can change the appearance of the room, induce apparent motion, extend the field of view, and enable entirely new game experiences.

Our system uses the appearance and the geometry of the room (captured by Kinect) to adapt the projected visuals in real-time without any need to custom pre-process the graphics. What you see in the videos below has been captured live and is not the result of any special effects added in post production.

  Microsoft IllumiRoom Concept