Full immersive 3D gaming around the corner

Microsoft file patent for full immersive gaming - They revolutionised gaming with the Kinect, now Microsoft are going beyond 3D and motion control with an application for a Holodeck style system.

In a patent summary, discovered by Patently Apple “User enjoyment of video games and related media experiences can be increased by making the gaming experience more realistic. Previous attempts to make the experience more realistic have included switching from two-dimensional to three-dimensional animation techniques, increasing the resolution of game graphics, producing improved sound effects, and creating more natural game controllers”.

Microsoft’s ‘Immersive Display Experience’ extends the viewing experience to the entire room via in-room projection and other devices, widening the peripheral environment of the game, so people can see enemies approach them from outside the screen. Microsoft believe the architectural features of the environment surrounding the bezel of a screen mute the entertainment potential of the media experience as they are out of context (a Sky Plus box wouldn’t fit within the environment of Call of Duty for example) and “the ability to perceive motion and identify objects in the peripheral environment (i.e., in a region outside of the high resolution display) may intensify the entertainment experience”.

A gamer using a console would see the usual primary image on the primary display (TV screen) and a projected peripheral image (environmental display) on room fixtures and ceilings such as the walls and surrounding furniture.

To enable such immersion, various devices are depicted. Left and right side facing RGB projectors are situated at the top of the screen although the projectors can be positioned elsewhere. The user may be able to increase the sense of immersion using suitable active 3D glasses synched with both the display and environmental projection.

Other 3D options include complimentary anaglyph glasses and parallax barrier/lenticular glasses free screens. In the glasses free 3D scenario, the parallax views of the projected images would be rendered in quick succession to accomplish a 3D display of the peripheral image via "wiggle" stereoscopy.

A depth camera situated above the screen would calculate the depth and position of the objects (including the gamer) in the room. This would allow projected images to be rendered correctly rather than distorted by environment fixtures, in a similar way to how 4D projection works.

Complementing this is a Kinect looking tracking device which can recognise the player’s movements and interpret them as game controls. It may also track a user's eyes to determine a direction of the user's gaze which could improve the appearance of an image displayed by an autostereoscopic display.

The combined depth camera and tracking device would enable some sophisticated features. Aside from ensuring geometrically correct projection on in-room objects, it could even recognise the colour of objects in the room and adjust the projected colour of the environmental display to ensure colours do not clash. It would also shield the user, effectively silhouetting them, so they are not distractingly projected onto themselves. For example, pixels in a DLP projection device may be turned off or set to display black in the region of the user's position.  It would also be aware of the user’s gaze so projected images can be distorted accordingly.

The patent was filed in March 2011 and there is no indication yet if it will be commercialised. However, the exciting news is that gaming is heading towards a more immersive future.

Whilst 3D gaming has not been the success expected, various devices are set to be launched to put people within the games, still using stereoscopic images. After being successfully funded after a Kickstarter project, the Oculus Rift headset will be launched commercially and could be a game changer (literally). Featuring stereoscopic 3D, a wide 110 degree field of view, low latency head tracking and two 640×800 screens, gamers will finally experience virtual reality gaming, so often predicted in the 1990's. Doom 3 BFG Edition was the first title to commit to supporting the device with their game, and it was revealed that Meteor Entertainment and their upcoming free-to-play shooter, HAWKEN, also agreed to support the Oculus Rift.

Sony are releasing a successor to their headset, set to be revealed at the Tokyo Game Show on September 20th. The new HMZ 3D gaming headset will introduce head-tracking by including a front facing camera, a feature lacking in the original which was more of a personal 3D media viewer.

Although not specific to gaming, the BBC R&D department have been working on a technology not too dissimilar to Microsoft’s system called Surround Video.
The goal is to “create a richer and more immersive experience for the viewer” by replicating the “awareness human peripheral vision provides".

As above, the TV acts as a primary display and projectors surround the outside of the TV with peripheral video. This extra information can be captured using a special wide angle camera or synthetic material that is generated automatically by processing the central video. Such synthetic material may display the ambient colours of the scene, motion cues derived from the original material. The peripheral video is projected to the back wall which is reflected to the rest of the room although it does not include the sophisticated geometry correction detailed in Microsoft’s above patent.

Exciting times are ahead!

 BBC R&D INTRODUCTION TO SURROUND VIDEO 

FREE WEEKLY 3D NEWS BULLETIN –