Gesture Technology: What It is and How to Use It at Your Next Event

Even if you haven't heard the term "gesture recognition technology," there's a good chance you have used it or will see it soon at events. The technology's most well-known application is Microsoft’s Kinect device for Xbox 360, which allows users to control and interact with the system without touching a controller. Since its 2010 launch, Kinect’s hands-free, sensor-based technology has been adapted for applications in medicine, automobiles, and the home. Brand marketers are also using gesture technology to engage consumers at events with entertaining, interactive experiences that also prompt users to share the experience on their social networks.

“It’s a really fun way of deploying activities at events. It’s very social,” said Erik Muendel, C.E.O. of Brightline Interactive, which has partnered with marketing agency Team Epic to create gesture-based solutions for brands such as Pepsi, Nissan, and AT&T. "We have found that people actually cheer for other people that are doing it, so it adds to the crowd getting involved and heightens the overall experience." Muendel said gesture-powered activities can be controlled by users up to eight feet away, which creates better engagement for onlookers than a touchscreen interface where the user stands very close to the screen, making it difficult for others to see what is going on.

At the N.F.L. Experience leading up to this year's Super Bowl, Brightline created a gesture-driven “Throw Like Drew (Brees)” football passing game for Pepsi. Fans stood in front of a screen and moved in a throwing motion; depth cameras tracked their movements and calculated the passing distance. Pepsi made a donation of $25,000 to two charities in New Orleans based on total passing yardage. “People were playing it all day long. It wasn’t just fun, it was a unique way of getting people involved and impacting the charity,” Muendel said.

Gesture technology can also be used in presentations. At a conference hosted by the Association of the United States Army, Brightline created a system that allowed a presenter to control a slide show displayed on an 80-inch screen simply by moving his hands. “The application tracked the palms of the presenter. It was a presentation to talk about the 10th anniversary of the war in Afghanistan. The presenter could push the timeline up and down in virtual space, stop it with their palm, hold over a key event, and then it would drill in and show media from that event,” Muendel said.

Particularly for entertainment-type applications, such as Pepsi’s game at the Super Bowl, gesture technology is integrated with social sharing options. The activations may include leaderboards or take a photo of participants in action, and all of that is instantly shareable. “We use the technology to get people engaged and create unique content, and then we provide them the tools to share that content with a goal of getting their data. To share it you have to connect in some way. The engagement grows exponentially post-event, and all of that is trackable,” Muendel said. The gesture activities can also be repurposed for a mobile app, so the experience can live on after the event.

In the coming months, Muendel expects gesture technology will become more precise and less expensive. “The next wave will not just map your skeleton—it will map your facial expressions,” he said. "It’s out there now, but it’s expensive. In the next six months to a year, there will be low-cost sentiment feedback. You will be able to know how people are interacting with your content at an event."