Mistaken Metrics
For the rest of the October 2009 issue of CRM magazine please click here. One article in particular, "Your Metrics Are Outdated," appeared alongside this one.
Tony Hsieh, chief executive officer for online retailer Zappos.com, told a funny story at a recent conference about the lengths to which his customer service agents will go.
A Zappos.com employee was away on a business trip, Hsieh recalled, and after a night out with friends and coworkers, the employee had a hankering for pizza. Unfortunately, the hotel was no longer delivering room service. Not knowing the area well, the hungry crew had a funny thought: Why not call their own company’s customer service line?
Sure, Zappos.com is known for its exceptional customer service—but usually that involves, you know, actual customers. A customer service rep (CSR) in the shoe etailer’s Las Vegas headquarters might be understandably confused by a call about a pizza delivery in some other city. But, as Hsieh told the tale, the customer service agent delivered—and then some.
Within minutes, the CSR was back on the phone sharing with his on-the-road colleague a list of late-night pizza joints in the area that were still delivering pies. Why? Because all Zappos.com employees — not just CSRs — are trained to go above and beyond. In fact, the company has an unconventional metric within its contact center operations called “The Wow Moment.” Rather than emphasizing such commonplace figures as average handle time and first-call resolution, the company targets harder-to-measure responses—those that involve gaping jaws, widened eyes, and joyous exclamations — which give rise to the metric’s unusual name.
As part of his tale, Hsieh scoffed at the contacts-per-unit (CPU) metric, currently in vogue, which most companies strive to keep to a minimum. Zappos.com tries to keep that CPU number high, and Hsieh had a very compelling reason why: “We want to talk to our customers.” The company prides itself on its unique culture, and has served as a model for others advocating customer experience and corporate transparency—so much so, in fact, that Zappos.com became a billion-dollar takeover target for Amazon.com.
So Zappos.com’s “Wow” metric works — for Zappos.com. The brand, with its consistently-83-or-higher Net Promoter scores, makes the effort look easy. But as most organizations know, delivering satisfaction while maintaining efficient operations, making sales, and staying afloat in a sinking economy is anything but easy. To generate their own kind of “Wow” moments, companies have quite the juggling act. Keeping track of measurements is essential — but which measurements? Pick the wrong set of metrics, and you’re setting up flawed relationships. (See Customer Centricity, page 12, for more on that.) Unfortunately, there are many ways your metric mojo can fail to match the Zappos.com model.
YOU'RE TOO WEAK TO 'WOW!'
Many experts in benchmarking and implementation agree that only the crème de la crème can measure “Wow!” in place of (or even in addition to) traditional metrics. Cheryl Coppens, a director at Customer Operations Performance Center (COPC), a group dedicated to contact center benchmarking, says that companies mature enough to start wowing customers should go for it. Most organizations aren’t there, though—and she says that many will never get there.
The majority of organizations have trouble measuring to begin with. Problems with metrics rise from flawed processes. “The key is that companies have already got the processes in place so they can move to the next level,” she says. “It’s an absolute disaster when I walk into a company and their dissatisfaction is 25 percent and they’re saying, ‘Now let’s ‘wow’ everyone,” Coppens says. “That degree of dissatisfaction is saying there’s something wrong with your processes.”
Coppens says that in her experience and in researching regression, customer satisfaction can be directly correlated to waiting time and issue resolution. If you can’t resolve the issue, but you throw a coupon or a freebie on top, chances are the customer’s satisfaction level has not risen. “To throw ‘wow’ campaigns on top and expect bottom-box people to move up because you’re nicer is not going to do it,” she says.
It’s tempting, of course, after hearing success stories from companies such as Zappos.com to want to throw out the daily drudgeries of traditional metrics. And Coppens notes that wowing a customer is what elevates loyalty to levels you hear about with companies such as Apple and Virgin America. Who doesn’t want to dazzle customers? Coppens and others maintain that companies must set the table before they serve the dessert.
YOU'RE AIMING LOW
David Raab, marketing consultant, prolific blogger, and the author of The Marketing Performance Measurement Toolkit, insists that most organizations are, in one way or another, measuring incorrectly. Either they have flawed processes underneath the actual measurements or they have skewed results to portray themselves in a better light.
Raab focuses on the marketing industry, but his statement is true of virtually any department: “Where marketers falter in terms of their metrics [is] that they’re often measuring things that are easy, but not truly appropriate.” (See Required Reading, “Measuring Your Marketing,” page 23, for more on Raab’s view of performance management; see sidebar, “Make Your Metrics Matter,” below, for his take on the business value of metrics.)
It all boils down to figuring out how high you want to jump, says Lior Arussy, the founder and president of customer experience consultancy Strativity Group. You can choose to benchmark yourself against a mediocre competitor — or you can challenge yourself to match up to the best of the best. “Measure against Starbucks,” Arussy advises. “Are you willing to put yourself against an ‘exceeding expectations’ benchmark? That’s what will create experiences.” Organizations often dance around metrics such as customer satisfaction, afraid of learning the real nitty-gritty truth. “You’re only hurting yourself,” he adds. (See Arussy’s Customer Centricity column this month, “You Are What You Measure,” for more.)
Raab, Coppens, and Arussy all blame the popular four- and five-point survey scales as ways companies avoid harsh truths. Arussy says he’s even seen companies give customers surveys with only four levels of satisfaction. In that case, the organization has a 50:50 chance of being a top-tier company in terms of customer satisfaction. The industry veterans agree that 10-point metrics are much more reliable to gauge a customer’s true feelings.
YOU'RE NOT MATCHING METRICS TO MANAGEMENT
“It’s really important to understand your emerging issues and tackle them very early and understand how prescriptive you need to be,” says Coppens, who has been helping contact centers establish metrics for more than a decade. In other words, match your metrics to your objectives.
Also, shape those metrics — or at least help people understand them — across different departments. Help Joe in accounting, for instance, understand how he can influence a Net Promoter score.
The best benchmark measurement, Coppens says, is management of variation, by which she means realizing that there’s truly no such thing as an average call. Managers and agents must recognize that variation is bound to occur—whether it’s the time of day, a change in season, or a fluctuation in the market.
The reason Zappos.com is so successful is that its culture is practically tattooed on employees’ foreheads. A company without a clear corporate vision will struggle to make its mark on customers, and will likely fail to hit whatever numbers it sets up for itself.
Variation, however, can’t be allowed to degenerate into chaos. Employees empowered with a certain level of autonomy—but not too much—will take adaptive approaches. Coppens says she sees this a lot, but organizations that give employees freedom without framework pay the price. “If you aren’t clear on the steps, and agents then make it up, then shame on you, you’ve got a process issue,” she says.
YOU'RE MAKING YOURSELF THE ONLY MIRROR
“It’s just like having an auditor for your finances,” Arussy says. “You can’t be your own auditor—you’ll never have the integrity.” This doesn’t mean that you have to hire a consulting firm to tell you how you’re doing. It does mean, however, that employees bring inherent bias to measuring their own achievements, so proceed with caution.
Andrew Edwards, managing partner at Web analytics consultancy Technology Leaders and cofounder of the Web Analytics Association, says that he often sees analytics implementations that have been done by internal folks or an agency that doesn’t specialize in analytics. “Technology Leaders witnesses that kind of thing where customers will accept the Web analytics that is presented to them…without much iteration or customization of the accuracy of the numbers.” It’s impossible for them to see beyond the implementation to envision what could be achieved.
Edwards says not only is it hard for companies to know and see what they could be doing in their operations, but it’s even harder for them to recognize errors. One mistake can lead to hundreds. “What ends up in those scenarios is a pattern of wild inaccuracy where if you were to act on those things, you would be acting on incorrect information,” he says. Technology Leaders’ practice involves going in and doing audits on how numbers were achieved.
YOU'RE NOT INVESTING
There seems to be a perpetual gap between saying and doing with metrics. Organizations might say they’re committed to extensive measuring, but when the time comes to allocate money and resources, the commitment quickly fades. “We’ll ask marketers if they want to do better measurement, and it’s always a high priority,” Raab says. “Well, if it was a high priority they would have done it by now.”
You would think that because of the economic recession, businesses would be constantly on the watchtower, keeping a close eye on dashboards and financial reports. However, it’s just not the case. “There’s this disconnect between saying you want to do better measurement and committing the funds to doing better measurement,” Raab states.
“People should be more committed to measuring properly because if you [do so] you can optimize and ultimately save money by being more efficient,” Raab says. “But that’s a long-term view and we are in a short-term mindset as businesspeople.” In fact, he says it will likely get worse, not better.
YOU'RE LOSING YOUR BALANCE
When it comes to the processes and motivations that underpin measuring, the pendulum swings in both directions, requiring a delicate balancing act to involve key stakeholders, bridge departmental silos, and keep the customer at the heart of matters.
“Many companies get too focused on measuring internal processes rather than the customer,” says Yacov Wrocherinsky, founder and CEO of Infinity Info Systems, a professional services company and Microsoft CRM channel partner. “They forget that, at the end of the day, customers are every company’s lifeline and it’s about learning how to serve them better.”
Blending external and internal metrics is especially an issue in the contact center. Coppens relays that it’s not enough for organizations to do customer satisfaction surveys. People should measure on their own too through tasks like call monitoring.
Another key balance organizations must strike is with short-term and long-term measurements. It’s incredibly easy to measure the short-term, Raab says, but comparing long-term metrics is what brings the most value and insight. You don’t want to wait until the end to get an idea of what your revenue is for the month or the quarter, says Chief Strategy Officer Matt Johnson of Innoveer Solutions, a CRM consultancy. On the flip side, Johnson says, “If you are measuring constantly and you can’t see any pattern because the measurements are too infinitesimal.”
Johnson says that organizations struggle not only with striking the right ratio of long-term to short-term metrics, but they also have trouble balancing the amount of data thrown an end user’s way. Johnson says a particular client comes to mind who complained that in his job there was so much stuff being reported on that it became noise. “He couldn’t get a sense for what’s meaningful,” Johnson says. “Is there such thing as too much measurement? In fact there very much can be.” Management must be careful in striking the right chord: Provide sophisticated analysts and users with enough data to make compelling conclusions, but at the same time, don’t overwhelm business users.
YOU ONLY PROMOTE NET PROMOTER
Arussy says the current trends in measurement are both wonderful and troubling. “Traditional thinking such as average handle time and internal metrics are taking a second chair to satisfaction measures.” As great as it is to include the customer’s voice, companies need to take a fair account of internal metrics versus external. Yet some organizations might be taking that to the extreme and replacing metrics with the simplistic, one-number answer to the “Would you recommend this company?’ question.
“One trend that scares me a bit is taking a [contact] center and applying the Net Promoter score as a replacement,” Coppens says. The Net Promoter score (NPS), a metric that basically revolutionized the customer satisfaction arena, debuted in 2003 and is used by companies to gauge customers’ inclination to recommend. However, analysts and metrics experts are hesitant to fully praise the scoring system.
Because NPS is a reflection of the company as a whole, Coppens advises contact centers to use additional customer satisfaction metrics to complement NPS. “It’s great that you can ask a customer, ‘What is your likelihood to recommend?’ Just remember that feedback is about the company, not the call center.”
Raab points out that NPS is an easy metric to understand — both from the consumer and organization standpoint. However, with that comes Raab’s criticisms: “Net Promoter…is a very simple metric,” Raab says. “Life is never so simple.” He adds that, because NPS isn’t diagnostic, a company may find out that a customer was dissatisfied but have no idea what went wrong or what may need fixing. “Even if it is a good top-line measure, it’s nowhere clear enough,” Raab says. (Contributor Paul Greenberg examined the world beyond NPS in his August 2009 Connect column.)
YOU'VE GOT YOUR NUMBER
No matter what you do or what you measure, you’re destined to pick the wrong yardstick at least once. The trick, experts say, is not to force yourself to live with the wrong measurement — the trick is recognizing when the measurement you’ve chosen is the wrong one and having the fortitude to step away from it.
.
SIDEBAR: Make Your Metrics Matter
David Raab, principal at consultancy Raab Associates, argues in The Marketing Performance Measurement Toolkit (see Required Reading, page 23) that marketers must identify what “success” means. Associate Editor Jessica Tsai asked Raab about attaching business value to whatever you measure:
"One of the distinctions I make in the book is the difference between measuring results—activities, efficiencies—and value. I can easily measure activities: how many clicks I got and the cost of those results. The trick is to measure the business value—finding what those clicks are worth gets tricky and that has to be interpreted in a larger context.
Metrics always have to be interpreted in context, which is usually performance against plan: Any measurement system has to have some sense of what you expected to happen so you can compare against it. That, in many ways, is more important than this absolute measure of business value."
Contact Assistant Editor Lauren McKay at lmckay@destinationCRM.com.
You may leave a public comment regarding this article by clicking on "Comments" at the top.
To contact the editors, please email editor@destinationCRM.com.
Every month, CRM magazine covers the customer relationship management industry and beyond. To subscribe, please visit http://www.destinationCRM.com/subscribe/.