Gesturing at GestureTek

GestureTek was a field trip that will be hard to beat! It was very informative and a lot of fun. The technology developed here could affect many areas of our lives, from entertainment, to rehabilitation to consumerism. They have found niches for their products in so many areas. And I’m sure they will find even more.

The engine behind all of their products is their patented ‘video-gesture-control’ system. The main idea behind this is the use of cameras for user input. Using cameras frees the user from having to use a mouse, keyboard, tablet, etc., for input. It allows them instead to use their entire body in some cases as an input device. In some cases the users motion merely affects a visual image that is projected onto a surface, in some the user is able to navigate through and manipulate content and in others the user is immersed in a game and can move around and interact purely through their own body movements.

Pictured here is an advertisement created by GestureTek for, you guessed it, Pepsi. GestureTek refers to this as ‘GroundFX’. This type of advertisement is very eyecatching and interactive. There is a camera above that projects the image onto the ground and also through the use of cameras any motions made over the projection are translated into a change in what is projected. In this case the Pepsi logo is shown in a pool of liquid. When someone waves a hand or foot or whatever over the image it responds by showing the liquid to move according to their motion.

I don't think there are many people out there who can resist a toy like this! I think that we live in such a commercial world and are exposed to ads and billboards so much that most of us have learned to just filter them out. Something like this may get past the filter though. It's eye catching and fun and interactive. Perhaps some day if we are inundated with this form of advertising we will learn to filter it out as well but for now it seems like a great way to catch people's attention. I do wonder what the next step would be after we do get used to this though!

Another interesting piece of technology they have is TableFX. These tables have a screen that, similar to GroundFX, projects a dynamic image that responds to a persons movement. These tables are in use at a restaurant in downtown Toronto called ‘Alice Fazooli’s’. Microsoft has also announced the release of a very similar table called ‘Surface’. Interestingly it would seem that this release may violate the patent held by GestureTek since it uses the same type of technology. When questioned about it we were told that it was something they would have to talk to Microsoft about.

Probably the most fun was had playing with their immersive game. While the game itself wasn’t particularly advanced in terms of gameplay and graphics, the method of input was certainly more fun than traditional keyboards, mice or joysticks! Playing the game involved standing in front of a green screen. There is a camera and monitor across from you. The camera picks you up and the program adds you to the scene on the monitor. Then the fun begins! In order to shoot down the targets you simply flicked your wrists in the direction you wanted to aim and in the game it looks like you are casting fiery spells from your hands! How fun is that? It’s much more immersive than traditional games. It could, if used with the right kind of game, even be much more physical exercise than your typical video game. This could be used not just for pure fun but also in rehabilitation. GestureTek has licensed the use of their technology to Microsoft for use in Xbox game development.

One of their other products that I would like to highlight is the projections on glass. Embarrassingly this is something I used to make fun of when watching CSI or Minority Report. Turns out the jokes on me and the technology does exist though I think the visual interface in CSI was a tad more advanced. Basically this is a screen projected on a pane of glass. There is a camera behind that picks up your movements and you can navigate through the interface based on this. Other companies are working on similar multi-touch interfaces. Perceptive Pixel’s main goal appears to be the creation of a multi-touch interface. Unfortunately their website doesn’t really say much about how this works underneath but I think perhaps it may be touch sensitive. While this is a different way of getting things done it is still definitely a gesture based mode of input. And it looks very slick!

Interestingly it is possible to do some of this through Flash as seen here where google earth is being controlled purely through detailed hand gestures. I'm looking forward to learning more about this in the very near future.

Visualization Design Institute and Immersion Studio at Sheridan College

We had our first field-trip at school today! We went to the far off reaches of the fourth floor of our building to the Visualization Design Studio. VDI is a computer visualization and simulation research group at Sheridan College. The two main areas of interest that came out of this trip were the facial recognition and the immersion studios.

The FACE (Facial Animation Communication Engine) research was quite impressive. You can see more detail about this project on the VDI website. In a nutshell the research aims to, using a simple webcam, recognize a face and track and recreate it's movement on a computer generated model. At this point they are able to track only basic head movements but as the research progresses the intention is to add the ability to track finer movements such as lip and eye motion. Having observed the work to date I was impressed with the lack of lag in the program. It appeared to be able to follow the subject without any noticeable delay. This could have use in the field of special effects and animation. It would simplify the motion capture process by eliminating markers and special suits that have to be worn by the actor. If the application is refined enough it could lead to extremely realistic and naturally animated facial motion. There have been some breakthrough’s in markerless motion capture but even this more refined method still involves covering whatever surface is being captured (be it skin or fabric or whatever) in some sort of phosphorescent ‘make-up’ in order for the process to work. The research started with the FACE project may at some point even do away with this requirement as it becomes more sophisticated and polished. I think there is entertainment value potential with this technology as well. From something simple like creating a customizable animated avatar for online VOIP/Webcam chats to a more interactive gaming environment. If this technology at some point is able to read fine and detailed facial movements then I imagine it can be extended to read entire body movements. The type of interaction now offered by the a game console such as the Wii could perhaps be extended even more to allow the user to interact with their entire body.

The now defunct Immersion Studio showed us how user interaction can be used with video branching to enhance the experience of something like a movie. Basically the viewer is given an input device (in this case a tablet PC) and while the movie plays at certain points the user is requested to make a choice that will affect the outcome of the movie. Based on this choice the program selects which branch of video to follow. Essentially it is the "Choose your own adventure" of the movie world. Partially due to the cost of the equipment needed to run this setup it seems to not have caught on. For some reason I keep envisioning this at an amusement park next to the “Action FX Theatre” where your seat moves (in theory) in response to what you see on the movie screen in front of you (frankly my experiences with this ride led more to motion sickness than to any sort of excitement and/or fun). On the less-fun-more-educational side I can see the immersion technology being extremely beneficial. It would be a great way to create tutorials that are more immersive and experiential. Branching the video to show the outcome of the users decisions could make for a very effective learning experience. Some of the potential uses of this technology can be seen here.

Would you dress your vacuum?

I found this very interesting...

roomba

I'm all for robots... but I don't know if I would dress up my vacuum...

k

My First Blog

Hi!