Direct eye contact video conferencing to be commercialised

The Fraunhofer Heinrich Hertz Institute in Berlin has made a breakthrough in direct eye to eye contact for video conferencing via a software module and two or three cameras. It is the result of years of research and the Fraunhofer HHI expect it to be implemented into consumer laptops.

As anyone who has used video conferencing applications will know, there is always the problem that each party is looking down to see the other party. Eye contact is one of the most important elements of non-verbal communication and without this, each person can feel slightly disconnected from the conversation (how many of you have given up and reverted to audio only?!?).

High tech R&D firm Fraunhofer Heninrich Hertz Institute in Berlin, Germany showed 3D Focus their latest work in solving this problem during the recent Future of 3D Media Day. Called the Virtual Eye Contact Engine, the software module analyses the scene in real-time 3D from three cameras mounted around the display. It computes the depth structure information of the person’s head which is used to generate a 3D model. The 3D model is then used to compute the view of the virtual camera for both parties and the rendered output appears to show each person looking directly at each other.

Such technology requires high quality video processing at high resolution but the Fraunhofer Heinrich Hertz Institute claim to have achieved real-time processing with low delay, HD image resolution and high quality rendered views.

The system could even work with two cameras making it more viable as a commercial product. The optimal number of cameras depends on the final application scenario and the expected quality. For example, the distance between the user and screen as well as the size of the screen influences the choice of the camera setup. The desktop prototype system shown to 3D Focus provides high quality HD virtual eye contact using three cameras instead of two as a third camera increases the quality and robustness.

The Virtual Eye Contact Engine was first shown at the 3D Stereo Media Summit in 2010 but has been considerably improved since then including the creation of the desktop prototype version (as shown by the pictures).

Ingo Feldman, Scientific Project Manager 3D Video & Immersive Media Group, told 3D Focus that he expects the solution to be commercialised, “We are working on a product release for our system. We expect the first version on the market in about one year’s time, i.e. autumn of 2012. We have two market segments. On one hand we plan to provide an office desktop system with high quality virtual eye contact correction. On the other hand, we plan a consumer market solution which is capable to run on laptop environments. As we are a research institution, we are always interested in industry partners which will finance our product development.”

Any camera can be used. Nevertheless, the quality of the generated virtual eye contact depends on the quality of the camera. For example, high quality HD processing will benefit from higher quality cameras. Ingo Feldman’s team plan to support a webcam based version too.

What is even more exciting is that the system can be adapted to work with glasses free stereoscopic systems. As an auto-stereoscopic display technology requires multiple views or video plus depth as input format, it perfectly fits to the output of the algorithms. The Virtual Eye Contact system is based on the 3D depth/structure estimation of the user in front of the screen so the required data formats for glasses free 3D video conferencing can be generated on the fly.

The Fraunhofer Heinrich Hertz Institute is currently working on the integration of the Virtual Eye Contact technology into professional 2D or 3D conference system environments. They also think further applications could evolve such as real-time 3D modelling of actors but this is still on-going research.

Ingo Feldman said “The feedback is in general very positive. Users and visitors in our labs and on exhibitions are enthusiastic to see that it is finally possible to solve the problem of missing eye contact for video communication. This has been a research challenge for more than a decade.”

Ingo Feldman and his team will be doing a presentation of the system at InfoComm in Las Vegas, June 2012.

VIDEO - VirtualEyeContact_1_CeBIT2011_640x480 (1)

FREE WEEKLY 3D NEWS BULLETIN –