When Disaster Strikes!
California constantly trembles in dress rehearsals for "the Big One." Midwesterners too often feel like they have gone to Oz-the part where only the wicked witch resides-when tornadoes flatten their communities. The Atlantic coast and Hawaii are deluged with water and wind in the face of hurricanes that bear seemingly innocent names.
Unfortunately, it's not only Mother Nature's wrath that can harm a business. Anything from a power failure to a network down or a mentally unstable coworker who knows just enough can screw up a system and take a company offline.
From a business owner's perspective, even a small interruption in service can result in a loss of revenue, customer confidence and loyalty. However, many companies, especially rapidly expanding businesses, aren't properly insured or otherwise prepared in the event of a disaster. Some people harbor a false sense of security that their company will never be beset by problems, but that laissez faire attitude can be costly.
Jim Lovvorn is the director of customer care for an AT&T customer interaction center (CIC) in the Seattle vicinity. The 800 people who work there are covered by disaster plans that incorporate the company's other half dozen CICs as well as five or six outsourcers to pick up the slack when there's an interruption-or overload-in inbound communication.
"We try to keep a balance between people, quality and profit," Lovvorn says. "The people are our employees, the quality is our link to our customers and the profit is our responsibility to our shareholders. The three entities are linked together. If we're not operational, we're not taking care of our customers. If we don't provide a safe place of employment, we're not taking care of our people, which will ultimately affect our profit. AT&T is committed to providing a safe and comfortable work environment for its people."
Designing From the Ground Up
If you're designing a new center, consider developing your disaster plan from the start. Paul Marshall, senior director customer support for Yellow Freight System (Overland Park, Kan.), the nation's largest less-than-truckload carrier, has been with the company for 15 years, originally as a branch manager of terminals. "As a less-than-truckload carrier," he explains, "we carry everything bigger than a FedEx or UPS package-general commodities of various sizes. There might be 10 shipments in one trailer and 30 in another." Yellow Freight's 25,000 employees work for a Fortune 500 operation that enjoyed 1998 revenues of $2.5 billion on its way to being voted one of the top 100 technology companies this fall by CIO magazine.
In 1994, Marshall was part of a team that moved the decentralized customer services into two call centers totaling 500 employees. The Des Moines, Iowa, center opened in 1995, and the Sioux Falls, S.D., center opened five months later. Both centers are operational 24x7.
As the centers were designed, Marshall kept redundancies in mind and ensured that there were emergency backup generators, a network of multiple fiber optic lines and back up routers. "We built in as many redundancies as possible," Marshall says, "but it does get kind of expensive." When the centers first opened they had no mission-critical applications. "I'd say the worst situation we could have faced at that time would have been a great inconvenience, but not catastrophic." However, the centers have since become the central places to call for pickups. "If a center goes down and people can't call for pickups, we could lose millions of dollars in a matter of hours," says Marshall. "That realization took us to the next level in terms of disaster recovery."
"We used to be able to live without providing service for 24 hours if we had to, but customer service is now so important that we can't live without it for one."
Short-Term Crises
Not everyone is so lucky to plan a disaster recovery when the center is being built. However, most companies can restructure their existing systems to build in more redundancies, such as increasing the number of servers and other equipment that are on standby. As Marshall says, "We used to be able to live without providing service for 24 hours if we had to, but customer service is now so important that we can't live without it for one."
Lovvorn's AT&T facility has three categories of disaster plans in place: short-, medium- and long-term. His short-term plan covers disruptions ranging from systems problems, power outages, fire alarms and bomb threats that might force short-lived down times or evacuations. In the latter event, he says, "First we get people out of the facility," he says, "then route the calls to a sister facility in Sacramento, Calif. We also have a panic button in several places throughout the facility. If they're activated for any reason, the calls are routed outside."
Mid-Range Problem
A mid-range problem disrupts or closes a center for one or two days, usually from a weather-related occurrence like heavy snowstorms. AT&T has contacted local hotels to see how much advance notice they would require for the company to book a block of rooms in the event of inclement weather. "At that point," says Lovvorn, "we solicit volunteers to come to work prepared to stay in a hotel if necessary, so that they could make it back to work rather than be stranded at home. This way, at least we have partial staffs at all times. In order to operate such a system, however, we need to take into consideration the availability of restaurants or caterers for our people."
Even a weather problem that doesn't directly hit a center can seriously overload and strain its resources. Prior to his job at AT&T, Lovvorn spent a decade managing reservations at the Sheraton Hotel in Raleigh, S.C. He says that when Hurricane Andrew hit Florida, there was a tremendous flow of people coming inland and the overload of calls to the hotel's CIC had to be reallocated to different parts of the country.
If one of Yellow Freight's centers goes down, the second location takes over as much of the workload as possible. Marshall's group worked with Yellow Services, the company's tech team, to develop a voice recognition message that could explain to callers why a center was inoperative, that their calls were being rerouted and there might be an unusual delay. Database routing would then determine which calls would go directly to a representative at the other center and which would be put in a queue.
Yellow Freight is also working with a vendor to develop a third electronic site that could handle calls using an automatic voice response system. Calls can be routed to the appropriate people following a voice prompt, such as "1" for pickup routes. "We're well prepared if something happens to one site," says Marshall. "Fortunately, it's usually only a network problem, something going bad in the equipment."
Yet the sites have had their share of non-network problems. In 1997, tornadoes forced the Des Moines center to be evacuated several times during one day. In another instance, the same center closed in the early afternoon because of a blizzard. "The transition to the other site was seamless to the customers except that we lost the ability to answers calls as quickly as we'd like," says Marshall.
"Using an outsourcer to keep calls answered can be a solution for emergencies. But don't wait until the disaster strikes to form the necessary relationship with such a firm."
Long-Range Problem
In this age of immediate gratification, being down for two or three days can qualify as a long-term disaster. If forced to, Yellow Freight is capable of opening another center outside Kansas City, Kan., and making it operational within 24 to 48 hours. Most of their systems are proprietary and can be reconstructed by the company's IT team.
When Hurricane Floyd hit the East Coast in September, the AT&T Orlando, Fla., site was closed for one day. The New Jersey site was also heavily flooded and the network sustained some damage. The AT&T network diverted calls to centers in the western United states, but Lovvorn warns that this only works well if the various sites have linked databases. "You must ensure that the company has access to customer databases in other parts of the country," he says.
A disaster planning group should maintain a list of employee names, addresses and phone numbers that is updated at least every quarter, preferably every month. In the case of Hurricane Floyd, although the New Jersey center thought the Friday closure would be a one-day event, the team made calls during the weekend to schedule people for overtime on Monday. The center wasn't even able to reopen that fast, but had it been able to, those staffers who didn't have personal disaster issues-or who had other family members who could care for things while they worked-would have been ready to man the phone lines.
When Hurricane Fran hit North Carolina several years ago, many Sheraton employees needed personal time off to take care of their own needs. As things started to settle down, but not quite to normal levels, people brought family members to work with them, including young children who needed to stay in the temporary day care facility set up on site. "The company tries to help its employees with their personal lives," says Lovvorn. "There was a break room where people could go watch TV, and employees could get water and ice from us. Because of all the flooding and power outages, ice was at a premium."
AT&T CIC employees have an internal IVR number to call for information to find out if a facility is open, if people need to report to another location or if they are needed for additional shifts. Employees also know which radio stations broadcast information to employees and the community at large in the event of a natural disaster. Last but not least, someone on the team is responsible for posting information on the doors to the building if it's accessible, to give information to any employees who show up for work.
"We're always open to moving people to other locations," says Lovvorn. "We look to volunteers first-and find warehouses to convert to temporary call centers. We've thought of every detail. We've got banks of portable toilets in the parking lots, food in our on-site kitchens and generators in all the facilities."
Other Options
Although some companies, especially those that already outsource a particular segment of their calls, could turn to an outside party for help when they encounter problems, it's not so practical for everyone. Yellow Freight representatives, who handle 41,000 to 45,000 customer contacts a day, go through eight to 10 weeks of training before being considered "minimally qualified" for phone calls.
Although there are a number of simple transactions, says Marshall, "the complexity of what we do is pretty high. Many people who visit us are surprised by that. A `Where's my shipment?' call can lead to a 10-minute investigation to see if the shipment was misrouted, if something was missing or a check on reconsignments and upgrades."
Sometimes calls stop deliveries enroute and have them rerouted to other parts of the country. Or someone could call for a rate quote. Sounds simple, perhaps, but in many cases it's not. Since the cost of a shipment is based on weight, determining a shipment's weight is often the first hurdle. Then there's the matter of getting the product item number from the National Motor Freight Classification. "Not a day goes by that someone doesn't ask me something I don't know," explains Marshall. "For example, there are five pages of electrical switches. It's not enough for us to be told that the items to ship are electrical switches. Are they for cars, trains, planes or something else? It takes a lot of time to determine rates based on classification. Part of the training our reps go through is learning where to go for such answers and how to work with our knowledge management system."
However, for the right company, using an outsourcer to keep calls answered might indeed be an appropriate solution. But don't wait until the disaster strikes to form the necessary relationship with such a firm.
"As needs arise, we invent the answers. We don't have someone else's best practices to follow."
Meeting Clients' Business Recovery Needs
With a name that evolved from Computer Discount 30 years ago, Comdisco, headquartered in Rosemont, Ill., got its start in the mainframe leasing business. Since that time, Comdisco has sold its mainframe portfolio to IBM and has evolved to a technology services company with core services that include IT control and predictability, managed network services and business continuity and availability services. Comdisco Continuity Services, which has been in business for 20 years, helps organizations ensure the availability of their systems, applications and data across data centers, networks, distributed systems and work areas. With more than 50 recovery facilities worldwide, Comdisco's availability services include alternate sites, work area, network and mobile recovery, advanced work area recovery, which provide for financial services, and item processing recovery. In addition, Comdisco offers a full range of availability services for the CIC.
"We're focused on helping companies build redundancy into their business continuity plans," says Bob Felch, director of business continuity products. "Companies can look to us to recreate their CIC for a number of different reasons ranging from a system failure through a natural disaster. We can help organizations protect their mission-critical call center technology that encompasses computer telephony integration, automated call distribution, interactive voice response and voice over Internet protocol."
The first steps in creating a business continuity plan are to identify business-critical functions that an organization can't exist without, estimating the cost of a disruption, as well as determining what the recovery times are for each critical function," Felch says.
Comdisco's Professional Services group can assist a company in designing and implementing its business continuity plan. If desired, Comdisco will examine a client's internal business process and can conduct a business impact analysis to evaluate the organization's business functions to determine what is critical to the company's revenue and customer service and the potential problems if such functions are interrupted. Organizations can then establish an optimized timetable for recovery. The client's technology is evaluated, as are its external service providers, and possible disaster situations are examined. Comdisco's team can also help organizations identify the criticality of agents with the company. For example, a business may have 200 agents but only 20 to 30 might be viewed as critical, at least for a short-term problem.
Another critical part of business continuity planning is testing the plan on a regular basis. "Annually testing your plan dramatically improves the likelihood of success," Felch says. "Testing can often reveal planning details that can make the difference between a smooth recovery versus a crisis. That's the whole idea behind testing-to find out what doesn't work and fix it. For example, some call centers may find during testing that they have a document of a disaster plan, but there's only one copy of the plan, and it's stuck in the production office, which they may not be able to get into if there's structural damage to the office."
For longer-term recoveries when relocating employees is not an option, companies can use mobile recovery services for their call centers. Space can be set up with modular trailers that can accommodate everything from a small, 20-seat environment up to a 250-seat configuration complete with custom workstations, phone equipment and office furnishings. Comdisco can rapidly deliver the mobile facility and configure it to meet an organization's specific business needs, complete with the hardware, including PCs, phones, T1 line access and dishes for networks to help a company resume operations as quickly as possible.
If a company calls on Comdisco, there is an initial declaration fee to implement their assistance and a daily usage fee. Comdisco works with organizations of all sizes, working with them to design a custom solution to match that company's specific business needs. Customers sign contracts and pay a monthly subscription fee that can range from a couple hundred dollars to hundreds of thousands dollars per month depending on the size and complexity of the technology being recovered as well as the organization's recovery time requirements.
"No compromising. No limits."
Such is the stated philosophy of Willow, formed in June 1997 and based in Miami, Fla. The company maintains that it has the perfect solution for a center in flux-CyberAgentCSRs.
Willow refers to itself as a cybercenter network and trains its agents to assist companies any time extra staffing is required, whether that's during daily peak hours, special promotions, the transition from a "standard" 9-to-5 workday to one that's 24x7, and holidays. Companies post their extra hours or number of agents desired on the network, and interested agents can sign up for the shifts. There are currently approximately 1,500 Willow-trained agents.
Willow recognizes that an "optimum" number of reps leaves a company with periods in which too many people are on hand for the quantity of incoming calls as well as periods in which too many calls are dropped because of inadequate staffing. Willow's CyberAgentCSRs can step in on an as needed basis to eliminate the period of overflow, thereby decreasing the company's optimum staffing level to one in which all in-house agents are actively working throughout their shifts, and CyberAgentCSRs handle the overflow. Since they are normally paid per call, the company can keep costs to a minimum.
Since CyberAgentCSRs are located nationwide, agent talent isn't limited to the available geographic pool near a customer interaction center. Also, agents speak a number of languages, so a company isn't limited to whatever foreign language is prevalent in its location. If much of a company's business comes from Miami, for example, but its customer interaction center is in Provo, Utah, it can use Willow's agents from parts of Florida, New York, California and Texas, where many residents are native Spanish speakers, to fill the void.
The company stresses that its agents are not home agents with the extra costs and control problems associated with them. These agents are independent contractors (1099 employees) who work for multiple clients. CyberAgentCSRs supply their own equipped offices and work for four to six clients on an as needed basis. Their shift average is two hours per client per day. Thus, companies seeking to replace or add one full-time inside agent could instead support four CyberAgentCSRs. And since they are used for both scheduled and emergency time slots, the company is never at a loss when the unexpected strikes. There is unlimited growth capacity without added capital costs. The system works because companies using Willow's agents pay a usage fee to access them, while the agents pay a monthly connection fee to the network.
Nor are CyberAgentCSRs an outsourcing operation. Willow's agents become direct employees of the companies they contract with. Those companies are responsible for hiring, firing, scheduling, supervising and setting pay rates. Willow's role is in training the agents in both verbal skills and use of the technology. Training begins at local colleges, then is picked up by starTutor, the network's digital video training program over the intranet. It is here that agents learn the individual protocols for each hiring company.
"The basic class is 80 hours covering the use of the computer system and network," says Caz Norwich, director of public relations. "Customer relations, how to conduct oneself with clients and how to be an independent agent-everything from taxes to business expenses. The applied class that follows is specialized for each individual client and can include taking orders. The length of the applied class varies depending on the difficulty of the technology and the product. It can be from three days to two weeks."
The Network Consortium trains Willow's agents; companies simply graduate them. "The client screens the agents because the client is the one who hires," says Norwich. "Obviously, CyberAgentCSRs who have done call center work in the past have the best chance of succeeding." Such agents do exist. Some former full-time agents simply left to work at home and have more flexibility with their hours, to enjoy the variety of working for a number of different companies, or because they moved away from their former employers.
The network infrastructure, provided by BellSouth and Nortel, can be easily linked to CyberAgentCSRs' home systems, and includes the appropriate firewalls and other security systems. The Network Consortium takes care of the technical installation of all communication links. The clients can schedule agents far in advance through the network's starMatic posting system, make real-time changes, and receive comprehensive status reports.
Willow claims that its agents reduce lost, abandoned and on-hold calls by at least 50 percent and that the lack of building, equipment, recruiting and training costs ultimately leads to a 10 to 50 percent reduction in call costs. Willow also expects that the use of its agents can also improve a company's internal customer interaction center by eliminating unwanted overtime, reducing turnover and increasing wages.
"We're doing very well," says Norwich. "We have 14 clients, nine of whom signed up this year alone. As needs arise, we invent the answers. We don't have someone else's best practices to follow."
Companies wishing to learn more can attend a monthly live demonstration of CyberCenters in Miami or arrange to visit one of Willow's clients' CyberCenters.
Wake Up Call
Customers aren't going to care whether a company opts to redirect its calls to its other centers, to an outsourcer, a CyberAgentCSR, or to temporary facilities. They're just going to want to get serviced in the best manner possible. Employees also will appreciate the semblance of structure and having clear-cut responsibilities after a disaster changes their work environment. And, of course, there's the bottom line. Companies with a solid business continuity plan in place-and the insurance to help them with costs-will ultimately win the battle to restore incoming revenue as fully and quickly as possible.
Therefore, consider this a wake-up call for companies that are lagging behind in their plans. They now have the opportunity to assemble a team that can develop a plan and set aside a day in which to test it. If it doesn't work to everyone's satisfaction, there will still be time to meddle with it until it does. As Felch says, "Disaster recovery is certainly an area in which prevention is better than the cure."