Middleware: Keeping Business Applications Connected

Without middleware, your spouse will leave you, your crops will wither, and your dog will die.

Do I have your attention now? Good. Forgive the hyperbole but mddleware's subliminal role in enterprise automation make it, as a topic of discussion, about as sexy as the u-joint under your bathroom sink. But know this: You will ignore this high-tech u-joint at your own peril. Middleware is the glue that holds your entire enterprise system together. Without it, the incredibly expensive CRM solution you've staked your career on probably won't survive.

In this article, we will answer a few questions about the what, why and how of this often-overlooked breed of software.

What is middleware? Middleware is the software that allows disparate applications to communicate with each other. For instance, a company might have an ERP system from Oracle and a CRM system from Siebel. Data from one application passes through the middleware pipeline, where it is translated into a format that the other application can use.

Middleware comes in hundreds of different varieties, and is not something you simply purchase. It may come already embedded as a layer on the application server, or it might be a component of an "open architecture" business application. Another option is to purchase a middleware solution from a vendor who will customize it to work with both your new and legacy applications. You can even write your own middleware code if you wish.

You should also know that the term "middleware" applies to function, not brand. Basically, any technology that helps applications work together falls under this catchall moniker.

Why should I care? Because when enterprise systems cannot share data, you waste resources duplicating that data in each application. Additionally, "[without integration] companies end up forcing users to flip-flop between legacy applications and new applications," says Gail Greener, senior director of marketing at Propelis, a middleware vendor specializing in e-business integration. "For call centers where people are operating in real time, it is worse for the operators because they don't have the latest information."

Greener's reference to "real time" is important. In e-businesses, customers, as well as employees, access the data generated and housed in enterprise applications. For all the obvious reasons, this data must be accurate, updated simultaneously through the enterprise and easily accessible. If the latest shipping information is not available online, but stuck somewhere in a siloed inventory application, your customer won't get the information he or she wants, and consequently, won't buy from you again. "The importance of real-time flow cannot be underestimated," says Nigel stokes, CEO and president of Canadian middleware vendor DataMirror. "In the future, people will grow even more impatient than they are now. They will demand real-time data."

So what does all this have to do with my CRM system? In customer-centric organizations, data now drives everything: the front and back offices, the supply chain and just about every other area of business. Compenies must get that data in real time from the CRM application to other points throughout the enterprise where it is needed, in a form that is readable by other applications. "You need a reliable mechanism for getting the data back and forth," says Sally Hudson, an analyst at IDC. "That is the function of middleware. It doesn't just connect. It actually transports."

If enterprise automation is finally a reality, then why can't CRM and other business software vendors just design their products to integrate with others? Within limits, some already do that. For instance, business software vendor PeopleSoft created its enterprise suite, PeopleSoft 8, with an "open architecture" design that the company says will resolve many integration issues. Siebel Systems also opened the architecture of several of its solutions. But, according to some in the middleware industry, such efforts are not enough. "CRM companies are not focused on integration," says Propelis' Greener. "They are all talking the talk and yes, they are making it easier. But they are not delivering a full middleware solution."

"Most CRM vendors look at it as 'we'll get you started, then you migrate the data and maintain it in the CRM system,'" stokes says. "That's nice, but it is not the way the world works. Changes that occur across all architectures need to be reflected in real time. IT is the middleware that integrates those applications and provides for real-time flow."

How exactly does middleware work? Although it comes in countless versions and varieties, middleware generally falls into two categories:

Data-oriented middleware--Facilitates information sharing between different applications. This type is the most widely used with CRM applications.

Process-oriented middleware--Enables the processing and integrity of transactions. It also facilitates security and the transfer of transactional information.

Both categories of middleware reside on a server as one of many application layers. On the server, data may pass from a CRM layer, through the middleware layer, where it is translated into a common language by one of many standard middleware interfaces, such as ActiveX, XML or Java. The middleware application then distributes the translated data into another application layer, such as ERP, in a readable format.

Are there any alternatives to middleware? Yes, you can hardwire applications together, which means you hire a programmer to write code that connects two specific applications, but only with each other. This option, experts say, often proves unsatisfactory. According to Sharon Ward, vice president of enterprise applications and CRM at the Massachusetts-based technology consulting firm Hurwitz Group, one of the beauties of middleware integration is flexibility. If you need to add new applications to your system in the future, or you need to alter your business processes, middleware will easily accommodate these changes. Not so for hardwiring. "If you do have to integrate, middleware will make the whole process a lot less painful than if you hardwire."

still, says Karen Boucher, executive vice president of technology research firm, The standish Group, in West Yarmouth, Mass., many companies still create middleware applications in-house. "The biggest competition to any middleware product is companies creating their own middleware. Like any technology, you can create needed services yourself, but this is laborious. It's also going to be different for every application. Standard middleware, on the other hand, creates layers of application services that allow you to make changes without having to write code."

Is middleware a new technology? Hardly. According to IDC's Hudson, middleware dates back to the 1970s--an era of massive mainframes and later, the client-server. "These weren't particularly scalable," she says. "It was difficult to get data off the mainframe."

According to Tyler McDaniel, director of application strategies at Hurwitz Group, mergers and acquisitions also created data access issues. "One of the primary business drivers behind the development of middleware was mergers and acquisitions. When companies merged, they found they had information stored in separate systems and locations but no way to share it."

Consequently, companies began to write their own code to connect these systems, and middleware was born. The need for system integration increased with the advent of multifaceted ERP suites and more advanced enterprise technologies.

Who makes middleware? While there are literally hundreds of software companies offering integration technology in one form or another, in the middleware industry there are generally gorillas and pure-plays. The gorillas are giant software companies such as IBM, Oracle, Microsoft, Sun, Sybase and Computer Associates, and typically offer integration technology as part of an e-business platform. For instance, IBM--the original middleware vendor--offers WebSphere, an e-business app-server platform that features a middleware layer designed to interface with leading business applications. According to IDC's Sally Hudson, the app-server model is the wave of the future. "For designing and deploying new applications, the way to go is the app-server platform. These platforms include the ability to tie in legacy applications as well."

The pure-plays focus solely on middleware, which some argue gives their products a technological edge over their larger, more diversified competitors. Pure-plays currently moving and shaking in the middleware industry are, among others, Tibco, Vitria, DataMirror, SeeBeyond, webMethods and Propelis. While each of these companies offers integration solutions, they differ in technology used and application focus. For instance, DataMirror focuses on real-time enterprise data delivery, an area integral to the success of CRM initiatives.

Since its birth in the 1970s, the middleware industry has grown in proportion to the growth of enterprise applications. IDC reported that in 2000, the middleware industry generated $3.4 billion in revenue, a figure expected to reach $6 billion by 2005. This reflects a compound annual growth rate of 12 percent per year between 2000 and 2005.

How much does it cost? The cost of middleware depends on the number of users, the amount of customization required and a host of other factors. Tyler McDaniel of Hurwitz Group estimates that large companies pay somewhere in the neighborhood of $500,000 to middleware vendors for an integration platform.

It may be helpful to consider the cost of achieving your business goals without middleware. Middleware vendors are quick to point out just how expensive and inefficient it is to create your own. "To write a single interface between ERP and CRM takes two weeks of design time, four weeks of coding, two weeks to test and another week to implement," says DataMirrors' stokes. "Coders make $800 per day, and for nine or 10 weeks, that comes out to roughly $36,000. At that cost, middleware would pay for itself in one implementation."

While middleware vendors have an obvious interest in downplaying the create-your-own option, the fact remains that implementations of CRM and other enterprise solutions are difficult and costly. Because vendor middleware is designed to interface with leading CRM applications, and because it is almost always upgraded regularly, it allows users to bypass a significant amount of implementation-related pain. "Middleware is an important part of almost any CRM implementation," stokes adds. "Up to 60 percent of implementation costs can go to interfacing. Unfortunately, that cost is often underestimated. People say, 'we'll just get the information from our mainframe.' That's easy to say, but it is not easy to do."

I'm planning a CRM implementation. What do I need to do about middleware? If CRM is an add-on to an already fully integrated enterprise system, then you probably won't have to do much. "Most CRM vendors have an embedded middleware product," Boucher says. "If you are going to buy CRM, you should ask what type of middleware the vendor uses, and then confirm that it will work within your system."

However, if you are coming late to the enterprise integration game and you still have '80s-era technology silos, you should work with an integrator to design and create a solid integration strategy. In it you will identify all touch points between the new CRM system and existing applications. Then search for a middleware vendor with expertise in your vertical, and with solutions that will allow your system to meet your present and future needs.

Above all else, an integration strategy should reflect your company's overall business strategy. "An integration system is not a solution," McDaniel says. "As a business decision maker, you should focus on what you need to accomplish, and how certain applications will help you accomplish it. In all likelihood, an integration implementation will enhance that strategy."