Graham Tootell a Senior Technology Consultant at Plextek Consulting looks at how a virtual reality gaming headset could make immersive telepresence a reality.
Telepresence is not a new concept but the idea that one can be immersed within a remote environment – to look around and interact with a room that they are not standing in – still seems far-fetched. However, a cutting edge virtual reality gaming headset may provide the missing piece of the puzzle in terms of hardware.
The Oculus Rift (OR) has revolutionised the world of gaming by bringing immersive reality to the industry at an affordable price. Looking beyond gaming it allows engineers to realise their own ‘real-life’ applications.
In doing so, the OR opens up the possibility for a concept I call immersive telepresence. Essentially, this is a video communications system and means of controlling it such that it simulates complete user immersion within a remote location.
The availability of appropriate hardware is not the issue. On the contrary, a state of immersive telepresence can be achieved with a tablet computer, an OR device and a steerable stereo camera that mimics a pair of human eyes.
Immersion in a remote location can be a difficult concept to grasp at first. A good example is to think about using a remote controlled car. You can see the car and control its movements using a remote control, but in order to be immersed in that experience and control the car from the point of view of a driver in it, you would need a camera in the car to see those images. You need to ‘be’ the camera, which is where the OR comes into play.
The stereo camera forms the remote head. Using the OR to generate the immersive effect and connecting it to the remote cameras, it is possible for the user’s vision to be effectively ‘transported’ to this location. The remote head can mimic the user’s head movements in such a way that when the user turns his/her head, the camera copies this movement to give the effect of the user looking around the remote room.
While immersive telepresence may seem like an outlandish concept it is not the hardware or technology that’s required that is the issue. It’s the reliability of wireless communications, image processing capabilities and security that are the obstacles to making it viable and useable.
Specifically, to use the OR to achieve a sense of immersive telepresence is associated with a number of significant challenges. The most important of these is latency. To make the mimicry of human movements by a remote camera a positive user experience relies on the ability to achieve minimal latency across the connection.
With conventional control devices, the user consciously manipulates some interface to achieve their goal and as humans we adapt to long latencies. For immersive applications, the experience needs to be accepted at the subconscious level and that means keeping latency in the few tens of milliseconds.
The consequences of a latency that is too high are varied. If a user is immersed at the subconscious level to interact with a remote location, the remote head must mimic their movements almost exactly as they would in real life. Otherwise, the user can suffer disorientation; nausea and the experience will be undermined.
Achieving bandwidth levels that are capable of dealing with the data transmission required in an immersive telepresence session is another significant challenge. Video communications are growing in everyday use and the quality continues to improve. However, it is a notoriously bandwidth-hungry process and if the network cannot cope with the data travelling over it, the user experience is poor.
More specifically, although the user feels immersed into the remote location when using the modified OR, the images they can see with what feels like their own eyes, are video images, which are processed and transmitted to the screen in their physical location. So, if the image is degraded by way of the bandwidth not being sufficiently high, there is a danger that image artefacts seen by the user will disrupt the user experience.
Despite these testing hurdles, the levels of latency and bandwidth can be resolved. By selecting the appropriate off the shelf hardware, the immersive telepresence phenomenon is already quite compelling. Latency and bandwidth can be reduced and increased respectively by developing bespoke sets of hardware that are tightly integrated. However, one has to bear in mind that when doing this, the cost needs to be kept to a minimum if immersive telepresence is to be favoured for industrial use.
Immersive telepresence has the potential to be the remote monitoring of the future to enable more patients to be at home in a safe and clinically friendly environment, reducing clinical burden and optimising follow up outcomes.
Of course, this depends on how effectively engineers can overcome the issues mentioned above. Drawing on its experience in wireless communications and image processing, Plextek is currently undertaking a project using the Oculus Rift to bring the latency down to a few tens of milliseconds and attempting to resolve the bandwidth issues.
