When Frank Panko launched A View From My Seat, a photo-sharing app that helps sports fans determine what the view will be like from various venue seats based on photos shared by a community, he needed greater insight into users' preferences and other analytics.
Having applied A/B testing techniques to the Web version first, he knew he needed to do the same for the app. When Panko learned of a mobile experience management platform being developed by a start-up called Artisan, he jumped at the chance to try it out.
"When you add a feature to an app or Web site, you want to know that you're putting the right feature in the right spot," Panko remarks. "Doing the tests can be tedious and hard, so when I heard about Artisan, I wanted to get [the software] as soon as possible."
The product, Artisan Optimize, enables users to test, enhance, and manage native mobile applications (apps that run on a device's operating system versus a Web-based app) without writing code or resubmitting it to mobile app stores.
One of the first tests Panko and his team did after implementing the tool last year was on the app's search box. The original layout was a search box on top of a blank screen. "We wanted to see if adding a tip under the box that let people know they could search by venue name, team name, city, or state would get them to do more searches on it," Panko explains. To his team's pleasant surprise, after the text was inserted, the number of searches shot up by 25 percent. "It turns out that adding something as simple as a tip really helped usage," Panko says.
On the flip side, another test showed that adding bright or animated graphics could have an adverse effect on users' activities on the site. Before using Artisan's platform, the rate at which visitors interacted with the app was "pretty flat," according to Panko. Within a month after A View From My Seat introduced the first set of enhancements, the usage rate increased by 22 percent and grew by approximately another 14 percent the following month. "What we think is happening," Panko says, "is people are enjoying the user experience more and are more inclined to use it."
One of the key benefits of Artisan Optimize, Panko notes, is that the platform allows him to execute numerous tests and quickly update the app's user interface. The company generally runs tests for two weeks before deciding which change, if any, to make permanent. Testing different features and adding those changes to the application on Apple's App Store without the help of a tool like Artisan's could take months, Panko says.
"I'd have to put out one version of the app, see what the reaction is, and then go back to the App Store a few weeks later with another version, resubmit it, and wait for Apple to approve it, and then repeat those steps once I've chosen a winner," according to Panko. "We're talking about a span of about two to three months to identify a better version and get it into the hands of the people." Panko says he plans to continue using Artisan Optimize to test and enhance the app. With Artisan's assistance, he notes, "A/B testing helps us make the right decisions, get rid of the junk, and improve our business."
Since implementing Artisan Optimize, A View From My Seat has seen:
- searches increase by 25 percent;
- the overall usage rate of its app increase by 36 percent over two months; and
- at least two months shaved from its workflow.