Could a simple webcam be enough for gesture control in VR?

Extreme Reality is on a pursuit to ‘motionise’ games but their technology has the potential to offer virtual reality gesture control and motion tracking without expensive hardware.

After a stint at Gamescom, Extreme Reality is now on the road to commercialising software that has been in their labs since 2005. The Israeli based software company aims to allow games developers to ‘motionise’ their games without the end user requiring any special hardware other than a webcam. Since September 30th, an SDK has been released which provides full body motion analysis via a webcam.

The company’s USP is being “the only company to provide full-body, software-based, motion analysis and control to any computing device or operating system via a standard camera.”

Talking to 3D Focus, Asaf Barzilay, VP Products and R&D, explains how it works:

“We have a platform agnostic solution that can be easily and widely deployed on a standard 2D webcam which only needs to be VGA and above.  Based on the RGB image we receive, our algorithm analyzes what the user is doing in xyz coordinates in every frame and this information is then used to create a live 3D virtual skeleton of the user in real time. It detects the joints, head, centre of mass, elbows, shoulders ribs etc at centimetre accuracy.  The xyz coordinates of each one of these joints can be very easily compared to the previous frames in order to get really quick and subtle calculations of what the user did.  From there we can take this technology into various different applications, not just games but we have also been looking at gait analysis, interactve experiences in museums, marketing, advergames, brand promotion and interaction of users in stores.”

Leap Motion may appear to be an obvious hardware competitor but Barzilay believes motion control with devices close enough to touch do not make sense…

“We have been focussing on the domain where the user is further away from the camera and cannot touch the devices, about 1.5 metres away or more (up to 5 metres according to website). We have been checking some interaction with smartphones where the user is very close to the phone or with laptops where the user is half a metre away but at the end of the day the user is still able to touch the device." 

So how does Extreme Motion compare in accuracy to the Microsoft Kinect?

“The end user experience is very similar to the kinect but without the hardware cost.” said Barzilay. “We see a person standing in front of the camera, moving freely, being detected and analysed.  Every centimetre of motion is detected. We have taken a few games titles initially built for the Kinect and managed to replace the physical system with our extreme motion software. The games play the same but they can now run of every PC and every iPad.”

One of the games Barzilay is referring to is ProRiders Extreme Edition by VTree Entertainment.  It has been converted from its original game format using traditional keyboard and mouse gameplay to full body motion control. The game uses a player’s body movements to drive the game play, riding a snowboard down snow covered mountains or doing their favorite tricks at mountain terrain parks, all while collecting coins and achievements.

“All of a sudden, instead of it being a very standard keyboard game it was a complete motion game.  We call it the purpose of motionising the game” said Barzilay.

Cick here for demo video

With virtual reality being very much the buzzword right now, it is not surprising that the company sees integration of Extreme Reality software having potential in a virtual environment. 

“Virtual reality still needs some user interaction. Imagine a webcam RGB image and in front of you are four buttons which the user can touch with his virtual hands – This is something we are already working on.” said Barzilay.

He also explained that the technology could be potentially used in the future for avatar creation.  So, in theory, as long as a user is facing a webcam, that information could be transmitted across the world to recreate an avatar in a virtual world seen by a user wearing a head mounted display. 

The company’s revenue model is planned to be two fold – a revenue share with app developers or, if a game is not chargeable to the end user (an advergame for example), the license would be incorporated into the production budget. 

Could a simple webcam be enough for gesture control in VR?

There are several games available with the most recent being the first iPad game – Go Dance by Sega.  Other games include (the incredibly difficult!) fitness/music game Beat Boxer, sports games ProRiders Snow Board and Top Smash plus the free game Pandamania.  A new racing game is expected to be launched soon.  

The company is currently running the Extreme iPad Challenge 2013 – a contest run for game developers to promote motion controlled games for iPad devices. The contest is open to game developers that wish to “motionize” their existing or new iPad games.  The submission deadline is November 15th with a top prize of $10,000 and free marketing on offer.

Visit http://www.xtr3d.com for more!
Source artice from 3dfocus