Measuring the Customer Experience

Today's complex Web applications are providing new products and services directly to consumers, typically without human interaction. Additionally, intense competition in industries such as financial services, telecommunications and retail, have made the customer's online experience a key differentiator. Senior management is now requiring that IT work with business managers to understand and improve the customer experience and its effects on the bottom line.

Customer Loyalty
Web customers require not only that the system is working, but also that the response time be quick. If they can't find a product on your site or it takes too long, they may click over to your competitor. The key to keeping the customer loyal is managing their online experience from start to finish. Shopping, checkout, shipping, customer care, returns and other online services all have to function seamlessly.

Are your customers faced with an issue like the one depicted in Figure 1?

Figure 1: This is an actual image from a real customer trying to purchase a $1,000 product online.

The odds are that when this customer was unable to complete his order, both this $1,000 sale and his future business went to the competition. In fact, the Technical Assistance Research Programs (TARP) estimates that only 1 in 26 unhappy customers will pick up the phone. If your IT organization is relying on customer support to understand the customer experience, it is only looking at four percent of the picture.

Outdated Methods
In the past, IT has relied on synthetic transactions created by robots to try and simulate what the customer will experience. Marketing has used Web analytics, such as system statistics or click stream analysis to attempt to piece together the customer's experience. Unfortunately, Web analytics only provides a historic view and synthetics are a poor substitute for understanding actual use.

The Real Solution -- Monitor the Actual Customer Experience in Real-Time
A complete solution should -- monitor traffic in real-time, detect defects, group defects into incidents, identify users, prioritize incidents based on business impact and then provide problem triage using actual data to provide a root cause probability analysis. The system would then automatically notify the help desk, as well as other enterprise wide tools. Additionally, the system should provide management reports on the health of the application. Key capabilities to keep in mind include:

Real-Time Monitoring -- Monitor the actual traffic created by real users as it happens.

Detect Defects -- Collect data on individual transactions and then aggregate common defects into incidents that can be rapidly analyzed.

Real User Correlation -- Identify the problem and provide the name of the actual user.

Incident Prioritization -- Prioritize based on frequency, the nature of the defect (a slow page is bad, but a missing page is worse) and the class of users that are negatively impacted.

Business Transaction Monitoring -- Organize traffic into business transactions such as login, place order and checkout to name a few examples.

Root Cause Analysis -- Identify the probable source of the problem to rapidly dispatch the incident to the appropriate group.

Automatic Notification --Notify the help desk and key individuals automatically when critical incidents occur.

Reports -- Provide Service Level and Trend Reports that show the number of successful transactions and performance.

How It Works
Industry analysts refer to this entire class of capabilities as: End User Experience Monitors. Typically, the end user monitor is implemented as an appliance installed on the network. End user experience monitors are usually plugged into a mirror port on a switch right behind a firewall. See Figure 2.

Figure 2 -- Location of the end user experience monitor on a switch behind the firewall.

The advantage of this method is that it doesn't impact system performance. The system now monitors actual traffic in real-time, analyzes HTTP/HTTPS requests, identifies users via their login name and tracks transactions as they occur.

Conclusion
For years, a communication gap has existed between IT and Marketing. Marketing is focused on increasing revenues by optimizing the effectiveness of business transactions, while IT targets the performance of the infrastructure. The promise of End User Experience Monitors is that now IT can measure the infrastructure through business transaction performance, creating a common language that everyone can understand.

About the Author
Frank Holsworth is director of product management for the Customer Experience Manager product line at CA Wily. Frank's extensive experience in ecommerce as the VP of Merchant Operations and VP Product Management for Shop.com; and as the Director of Global ecommerce in CA's Consumer Division gives him a clear understanding of what IT and marketing managers face in optimizing their customer experience.

Related Articles

How to Connect with Business Transaction Management

BTM is heating up 2010.

Customer Experience Is A Key Differentiator

Research by Strativity Group shows that higher investments in customer experience result in lower attrition and higher referral and customer satisfaction rates. Cutting back is not an option. "The race is on," says Lior Arussy, president of Strativity Group.

ClickFox Analyzes the Customer Experience

The vendor's packaged solutions target three specific pain points in the contact center.

The Unseen Force Behind Customer Experience

destinationCRM 2008: The missing link in providing quality service is an effective contact center manager.

Conflict Cripples the Customer Experience

destinationCRM 2008: The push to truly provide a differentiated service experience must take into account the consumer's ecosystem—not the companies'.

CRM Versus CEM

Ventana Research's latest benchmark study lays out the difference between the two, and suggests why the "E" might have more future success than the "R."