Gesturing at GestureTek

GestureTek was a field trip that will be hard to beat! It was very informative and a lot of fun. The technology developed here could affect many areas of our lives, from entertainment, to rehabilitation to consumerism. They have found niches for their products in so many areas. And I’m sure they will find even more.

The engine behind all of their products is their patented ‘video-gesture-control’ system. The main idea behind this is the use of cameras for user input. Using cameras frees the user from having to use a mouse, keyboard, tablet, etc., for input. It allows them instead to use their entire body in some cases as an input device. In some cases the users motion merely affects a visual image that is projected onto a surface, in some the user is able to navigate through and manipulate content and in others the user is immersed in a game and can move around and interact purely through their own body movements.

Pictured here is an advertisement created by GestureTek for, you guessed it, Pepsi. GestureTek refers to this as ‘GroundFX’. This type of advertisement is very eyecatching and interactive. There is a camera above that projects the image onto the ground and also through the use of cameras any motions made over the projection are translated into a change in what is projected. In this case the Pepsi logo is shown in a pool of liquid. When someone waves a hand or foot or whatever over the image it responds by showing the liquid to move according to their motion.

I don't think there are many people out there who can resist a toy like this! I think that we live in such a commercial world and are exposed to ads and billboards so much that most of us have learned to just filter them out. Something like this may get past the filter though. It's eye catching and fun and interactive. Perhaps some day if we are inundated with this form of advertising we will learn to filter it out as well but for now it seems like a great way to catch people's attention. I do wonder what the next step would be after we do get used to this though!

Another interesting piece of technology they have is TableFX. These tables have a screen that, similar to GroundFX, projects a dynamic image that responds to a persons movement. These tables are in use at a restaurant in downtown Toronto called ‘Alice Fazooli’s’. Microsoft has also announced the release of a very similar table called ‘Surface’. Interestingly it would seem that this release may violate the patent held by GestureTek since it uses the same type of technology. When questioned about it we were told that it was something they would have to talk to Microsoft about.

Probably the most fun was had playing with their immersive game. While the game itself wasn’t particularly advanced in terms of gameplay and graphics, the method of input was certainly more fun than traditional keyboards, mice or joysticks! Playing the game involved standing in front of a green screen. There is a camera and monitor across from you. The camera picks you up and the program adds you to the scene on the monitor. Then the fun begins! In order to shoot down the targets you simply flicked your wrists in the direction you wanted to aim and in the game it looks like you are casting fiery spells from your hands! How fun is that? It’s much more immersive than traditional games. It could, if used with the right kind of game, even be much more physical exercise than your typical video game. This could be used not just for pure fun but also in rehabilitation. GestureTek has licensed the use of their technology to Microsoft for use in Xbox game development.

One of their other products that I would like to highlight is the projections on glass. Embarrassingly this is something I used to make fun of when watching CSI or Minority Report. Turns out the jokes on me and the technology does exist though I think the visual interface in CSI was a tad more advanced. Basically this is a screen projected on a pane of glass. There is a camera behind that picks up your movements and you can navigate through the interface based on this. Other companies are working on similar multi-touch interfaces. Perceptive Pixel’s main goal appears to be the creation of a multi-touch interface. Unfortunately their website doesn’t really say much about how this works underneath but I think perhaps it may be touch sensitive. While this is a different way of getting things done it is still definitely a gesture based mode of input. And it looks very slick!

Interestingly it is possible to do some of this through Flash as seen here where google earth is being controlled purely through detailed hand gestures. I'm looking forward to learning more about this in the very near future.