January 1, 2015
By Maria Minsker, contributing writer

Is Hadoop Worth the Hype?

warehouse data. "I can pull data from both sources to run a combination analysis, to zero in on the customers I want, and then get the marketing folks to hit those customers with extra promotions," he said.

Hadoop Meets CRM

Though there are numerous solutions from both up-and-comers and legacy providers that promise to help businesses with the data migration and integration, operationalizing the data for marketers is still one of the biggest challenges that organizations face, and vendors are working to develop more marketer-friendly solutions.

"Often, marketers don't know what to do with the data they have in front of them. They don't have the specialized IT or analytics skills needed to extract insights from the data, but they know that the potential is there. What we know for sure is that using analytical models on systems like Hadoop that allow for faster extraction of insights can help us anticipate the future, and as a marketer, that's exactly what I want to do," says J. Graeme Noseworthy, senior director of marketing at RapidMiner, a data analytics and data mining platform provider. "I want to make predictions. I don't want to go off of old data or what the customer did, I want to know what he'll do," he adds.

RapidMiner's approach to Hadoop has been to simplify it, eliminate the need for end users to code, and do for Hadoop analytics what Wordpress did for Web site building, according to Noseworthy. The company's solution enables marketers to do the work of data scientists—that includes building data sets, moving them, and analyzing them at high speeds both locally and in the cloud—without any coding. At its core, RapidMiner's aim is to help marketers determine what Applebaum calls "the right questions" by making it easier to ask them.

Usually the work of data scientists or IT specialists, RapidMiner offers prebuilt data models—programs that query data based on certain conditions or requirements—for different data tasks or questions that marketers want answered. Predicting customer churn is one example of a RapidMiner model, though there are many others, such as customer lifetime value estimation, "next best action" models, and cross-selling models, according to Noseworthy.

"Without knowing what you're looking for, trying to make sense of a massive amount of data is like looking for a needle in a haystack," Noseworthy says, echoing Applebaum's concern about unfocused data exploration. "But analytics is an iterative process, and the more marketers do it, the better their insight becomes. By prebuilding the models that marketers can run themselves, we're letting them do what they love to do—try, test, and refine, but in a focused, action-driven way," he adds.

Once usable insights are collected, RapidMiner can connect the data platform to a marketing automation system or other digital experience management system to deploy campaigns or make changes based on data predictions. "We're connecting analytics to action," Noseworthy says.

PayPal used RapidMiner's solutions to get a better handle on predicting and reducing customer churn. The company applied RapidMiner's text-mining model to feedback it received from customers in more than 60 countries and was able to identify the top promoters and detractors of the organization. The solution, which analyzes historical and real-time customer data, predicts what may prompt a customer to abandon the brand, and uncovers specific customers that have expressed their intention to abandon, delivered some revealing results.

"Every month, we use RapidMiner for sentiment analysis of customer comments and feedback from around the world. It takes three days to analyze the text from over 150,000 customer post-contact comments, in almost every language, including 50,000 tweets and some public Facebook posts. What we find is that out of millions of active PayPal users, a sizable number of customers have password problems when trying to use PayPal," Jiri Medlen, senior text analytics specialist at PayPal, said in a case study.

Once PayPal asked its engineering team to make several improvements, the password recovery success rate went up by roughly 50 percent and password-related complaint volume declined significantly, according to the case study.

"PayPal was able to take an overwhelming amount of data and figure out exactly what was causing most of their problems in terms of customer churn," Noseworthy says. "For them, it was a matter of making some changes to their log-in process, but for other companies, it might entail cutting down on marketing emails or offering better promotions. Marketing is now all about the customer journey—finding out where things have the potential to go sour and where there are opportunities for great engagements. We're helping marketers do all this without a Hadoop headache," he adds.

Moving forward, analysts predict that leveraging Hadoop's potential will become a more attainable goal for companies. Because it's open-source, the possibilities are vast, Applebaum says, and Hadoop's ability to connect openly to other systems and solutions will increase adoption in the coming months and years. "There are going to be more and more vendors out there offering solutions that make transitioning to Hadoop and processing data on Hadoop fast and seamless, and the legacy vendors are going to have something to say as well," he says.

This is going to be the year of Hadoop, analysts predict. "The jury is in," Forrester's Gualtieri wrote in his report. "Hadoop has been found not guilty of being an overhyped open- source platform."

Associate Editor Maria Minsker can be reached at mminsker@infotoday.com.