The future of 3D video conferencing?

3D Focus, Dimenco, Triaxes, Tomsk Polytechnic University and Ravensbourne, successfully conducted a groundbreaking glasses free 3D conference on Thursday, connecting delegates over 4000 miles away in a futuristically immersive way.

Part of the Research Strand during the Ravensbourne 3D Storytelling event, the trial set out to provide a public glimpse of what video conferencing could be like in the near future. The audience in Tomsk, Russia were captured by a Panasonic AG-3DA1 camera and the London audience were captured with two Panasonic P2 cameras installed on a 3D Factory mirror rig.

Each side-by-side stereo feed was converted into a 2D + Depth Map (2D+Z) video stream in real-time by Triaxes NetJet software which enables stereo content to be displayed on autostereoscopic lenticular screens. The HD video and accompanying depth maps were transmitted in both directions, over 4000 miles to London/Tomsk. At each end, a Triaxes 3D Media Player received the IP broadcast, de-multiplexed and decoded the audio/video streams, added special control information (3D effect settings) and then sent the 2D+Z feed to a 42” Dimenco glasses free 3D display in London and a 55” display in Tomsk. The Rendering Cores integrated into the Dimenco displays then calculated the data and rendered out 28 viewpoints from the original 2D+Z stream.

The video was transmitted as a MPEG TS (transport stream) – the method commonly used in TV broadcasting.

Due to limitations of the IP channel, a low bitrate of 6.5 Mbits was used although Triaxes CEO Alexey Polyakov would like to see this be improved in the future saying “In the further demos we could use more complex compression methods which would provide better video quality with the same or less bitrate. Such a system would require more powerful computers on both sides.”

 THE TRIAXES SET UP 

During the 30 minute session, Triaxes Technical Officer Vlad Sidash spoke about how the system could be made compatible with a regular 3D channel feed such as Sky 3D, for output and real-time conversion to a lenticular format. Triaxes Manager Marina Pinzhina spoke about the technology and the requirement for generation of multiple frames.

Guests in the audience took the opportunity to ask their new friends about the differences between the Russian and UK media and Head of TPU Department on International Scientific and Educational Management Mazurina Olga, expressed her keenness to form closer links with Ravensbourne in London, as both Universities share a reputation of technological innovation and creativity.

Once the connection was closed, the Triaxes team in Russia surveyed the audience and we can now reveal the audience feedback. Marina Pinzhina took the opportunity to gage people’s perception, summarised as impressive but with room for improvements. She said "The glasses-free 3D conference was something really new in Tomsk that generated a great response from the audience. We had over 60 people from the university, government and media, when the expected number was 25 only. So it was hard to find the suitable viewing point for some of them. That is why we could not count the survey results as absolute one. Still the general opinion was rather positive.” 

Overall, the experience was evaluated at 6.5 out of 10.

The audience experienced the effect (especially those who were close to display).

50% said that it offered a significant difference to a Skype experience.
Respondents said…

-           The bit rate and picture definition would need to be improved.
-           It is the future of video conferencing.
-           The screen would need to be larger to make it compelling.
-           I do not like the fact there is no direct eye contact.
-           I would like such a system in my home.

Several of the above issues would be reasonably simple to solve. Bit rates could be improved with faster Internet connections but the lack of picture resolution is currently a major issue with lenticular displays in general. 3D Focus selected Dimenco screens for the trial as they offer the greatest number of viewpoints on the market (especially suitable for group public display) but, due to the way all lenticular screens work, resolution is considerably reduced compared to regular displays. However, Dimenco and others will soon be launching 4K versions of their screens to greatly improve the image quality. 3D Focus agrees that larger screens are required to make the experience more compelling and for public 3D video conferencing a 55” diagonal would be a minimum size.

Again, we agree the direct eye contact issue is not ideal, even on regular Skype video chat. However, this is set to be resolved soon. The Virtual Eye Contact engine, created by the Fraunhofer Heinrich Hertz Institute, is a software engine that analyses the scene in real-time 3D from three cameras mounted around the display. It computes the depth structure information of the person’s head which is used to generate a 3D model. The 3D model is then used to compute the view of the virtual camera for both parties and the rendered output appears to show each person looking directly at each other. The technology is planned to be commercialised in laptops. Dimenco are also working on direct eye contact algorithms.

Possibly the biggest frustration during the experience was a four second delay, causing difficulties for flowing conversation. When Vlad Sidash was asked how the delay could be reduced he responded with "Indeed the delay of 2 seconds (in one way) is too much for videoconferencing, but it is not much if it were being used for TV broadcasting. We need to take into accont the distances and the fact that we used a  usual Internet connection and not all stages were controlled by us. Of course, in the local systems we could achive much a better delay. For example, at our , we had a delay of less than a half of a second." He continued… "As it was our first experience of international 3D videoconferencing I guess we got rather good results. In the future we are going to improve our system by using more advanced compression algorithms, less delay and our two-depth tehnology."

The live 3D teleconference between Tomsk and London was a fascinating experience and whilst many pundits debate the future of 3D entertainment, it is expected that 3D teleconferencing does have a very bright future. It genuinely adds immersion to the experience and what the Ravensbourne/TPU trial proved was that, with some tweaking, international glasses-less 3D videoconferencing is not far away. In fact, it is a bizarre experience – even with low resolution, it can appear that the screen is a window, bending time and space. Once that is larger and can be achieved in high definition, it might generally offer an alternative to long distance travel.

Speaking to Tomsk TV, Alexey Polykov said  "According to optimistic estimates, in 2-3 years will be dominated by technology, which will cease to use the glasses for viewing 3D effect".

Of course, this set up is not just restricted to videoconferencing. The Triaxes DepthGate solution can convert two image stereo content to multiview format for lenticular screens for displaying presentations, public display advertising. As DepthGate is a post process, there are not the demands of real-time conversion and the improvement of the quality reflects that. It also automatically corrects colour and aligns input video for both 3D glasses and glasses free. Plus, using two depth maps rather than one results in a more accurate autostereoscopic 3D effect.

In order for glasses free 3D to become a success, conversion services such as the Triaxes solutions will be critical – the alternative is to film new content with up to 8 cameras – not exactly cheap or practical!

Elecard group will be exhibiting at NAB next week showing a glasses-free 3D presentation system by Triaxes & Dimenco. If you would like to know more visit www.triaxes.tv/uk. If you would like to know more about Dimenco visit www.dimenco.eu. 

To watch a video feature from Tomsk TV click here.
Installation photos can be seen .
To see photos from the live event click .

 TRIAXES PROMO VIDEO 

FREE WEEKLY 3D NEWS BULLETIN –