Back in January we’ve first introduced to you IllumiRoom which is “a proof-of-concept system that augments the area surrounding a television with peripheral projected illusions to enhance traditional gaming viewing experiences.”

The project was presented earlier this month at the Conference on Human Factors in Computing Systems in Paris. Unfortunately, we could not obtain the submitted paper detailing IllumiRoom but more information about the system has surfaced around the time of this conference. So check out the official gameplay video!

At first glance, this proof-of-concept augmented-reality system appears straightforward: create an immersive experience by integrating an off-the-shelf projector with a Kinect Sensor to extend the field of view (FOV) surrounding the display. It takes a few moments to realize that IllumiRoom is extrapolating from images on the screen: The projections are being rendered in real time in response to on-screen content. Delve into the research behind the project, and it turns out the IllumiRoom team not only solved immense technical problems, but also had to cope with creative challenges to blur the lines between the physical and the virtual.

Hrvoje Benko,  a researcher with the Natural Interaction Research group, comments on IllumiRoom:

Ultimately, it’s the content which makes the experience shine. We spent a lot of effort designing different ways to use content from games or movies specifically for this setup. We tried different approaches, from intercepting controller commands and using computer-vision techniques to recording new custom content. They were all challenging, and some worked better than others. Those were the lessons we needed to learn.

So far, we’ve made the experience spill out into the room, but in the future, we’d like to make it much more interactive. For example, if a ball falls out of the screen, we’d love to be able to throw the ball back! Plus, we’d like to understand what new cinematic effects are possible when content is split over two connected screens.

Source: Microsoft Research