Middleware: Directing Enterprise Traffic
Without middleware, your spouse will leave you, your crops will wither, and your dog will die.
Do I have your attention now? Good. Forgive the hyperbole but the very mention of the word "middleware" often induces sleep in even the most ardent technology groupie. Middleware's subliminal role in enterprise automation--it is often referred to as "plumbing"--and its amorphous nature make it, as a topic of discussion, about as sexy as the u-joint under your bathroom sink. But know this: You will ignore this high-tech u-joint at your own peril. Middleware is the glue that holds your entire enterprise system together. Without it, your dog may live on, but the incredibly expensive CRM solution you've staked your career on probably won't.
With this in mind, there are a few things you should know about middleware. In this article, CRM
magazine will answer a few common questions about the what, why and how of this often-overlooked breed of software. We will then examine the increasingly important role it plays in the success of both CRM and enterprise automation initiatives.
What is middleware? Middleware is the software that allows disparate applications to communicate with each other. For instance, a company might have an ERP system from Oracle and a CRM system from Siebel. Data from one application passes through the middleware pipeline, where it is translated into a format that the other application can use.
Before you rush out to Office Depot and ask for the latest off-the-shelf middleware miracle, be aware that this breed
of solution comes in hundreds of different varieties--many of them homemade--and is not something you simply purchase and download to your hard drive. It is complex and often expensive. It may come already imbedded as a layer on the application server, or it might be a component of an "open architecture" business app. Another option is to
purchase a middleware solution from a vendor who will
customize it to work with both your new and legacy
applications. You can even write your own middleware code if you wish.
You should also know that the term "middleware" applies to function, not brand. Basically, any technology that helps applications work together falls under this catchall moniker.
Why should I care? Because when enterprise systems cannot share data, you waste time, money and energy duplicating that data in each application. Additionally, users may be forced to leave one application to access the data stored in another. "[Without integration] companies end up forcing users to flip-flop between legacy applications and new applications," says
Gail Greener, senior director of marketing at Propelis, a
middleware vendor specializing in e-business integration. "For call centers where people are operating in real time, it is worse
for the operators because they don't have the latest information."
Greener's reference to "real time" is important. In
e-businesses, customers, as well as employees, access the data generated and housed in enterprise applications. For all the obvious reasons, this data must be accurate, updated simultaneously through the enterprise and easily accessible. With the growth of self-help, the importance of
real-time data access only increases. If the latest
shipping information is not available online, but stuck somewhere in a siloed inventory application, your
customer won't get the information he or she wants, and consequently, won't buy from you again. "The importance of real-time flow cannot be underestimated," says Nigel stokes, CEO and president of Canadian middleware vendor DataMirror. "In the future, people will grow even more impatient than they are now. They will demand real-time data."
So what does all this have to do with my CRM system? As mentioned above, data now drives the enterprise. Arguably the most important data a company owns is the highly personalized customer information that CRM solutions generate. In customer-centric organizations, data now drives everything: the front and back offices, the supply chain, product lifecycle management, inventory and just about every other area of business. The problem then becomes getting that data in real time from the CRM application to other points throughout the enterprise where it is needed, in a form that is readable by other applications. "You need a reliable mechanism for getting the data back and forth," says Sally Hudson, an analyst at IDC. "That is the function of middleware. It doesn't just connect. It
If enterprise automation is finally a reality, then why can't CRM and other business software vendors just design their products to integrate with others? Within limits, some already do that. For instance, business software vendor PeopleSoft created its enterprise suite, PeopleSoft 8, with an "open architecture" design that the company says will resolve many integration issues. Siebel Systems also opened the architecture of several of its solutions. But, according to some in the middleware industry, such efforts are not enough. "CRM companies are not focused on integration," says Propelis' Greener. "They are all talking the talk and yes, they are making it easier. But they are not delivering a full middleware solution."
"Most CRM vendors look at it as ‘we'll get you started, then you migrate the data and
maintain it in the CRM
system,'" stokes says. "That's nice, but it is not the way the world works. Changes that occur across all architectures need to be reflected in real time. IT is the middleware that integrates those applications and provides for real-time flow."
How exactly does middleware work? Because there should be a warning about operating heavy machinery while reading the answer to this question, we'll try to keep it short and to the point. Although it comes in countless versions and varieties,
middleware generally falls into two categories:
Data-oriented middleware--Facilitates the sharing of information between different applications. This type is the most widely used with CRM applications.
Process-oriented middleware--Enables the processing and integrity of transactions. It also facilitates security and the transfer of transactional information.
Both categories of middleware, along with myriad subcategories, reside on a server as one of many application layers. On the server, data may pass from a CRM application layer, through the middleware layer, where it is translated into a common language by one of many standard middleware interfaces, such as ActiveX, XML or Java. The middleware app then distributes the translated data into another application layer, such as ERP, in a readable format.
Are there any alternatives to middleware? Yes, you can hardwire applications together, which means you hire a
programmer to write code that connects two specific applications, which will be able to share data with each other, but only with each other. This option, experts say, often proves unsatisfactory. "The more you hardwire your business process, the less likely you are to respond to differing business situations," says Sharon Ward, vice president of enterprise applications and CRM at the Massachusetts-based technology consulting firm Hurwitz Group.
According to Ward, one of the beauties of middleware integration is flexibility. If you need to add new applications to your system in the future, or you need to alter your business processes, middleware will easily accommodate these changes. Not so for hardwiring. "If you do have to integrate, middleware will make the whole process a lot less painful than if you hardwire."
still, says Karen Boucher, executive vice president of technology research firm, The standish Group, in West Yarmouth, Mass., many firms still create middleware applications
in-house. "The biggest competition to any middleware product is companies creating their own middleware. Like any technology, you can create needed services yourself, but this is laborious. It's also going to be different for every application. Standard middleware, on the other hand, creates layers of application services that allow you to make changes without having to write code."
Is middleware a new technology? Hardly. According to IDC's Hudson, middleware dates back to the technological ice ages--the 1970s--an era of massive mainframes and later, the client-server. "These weren't particularly scalable," she says. "It was difficult to get data off the mainframe."
According to Tyler McDaniel, director of application strategies at Hurwitz Group, mergers and acquisitions also created data access issues. "One of the primary business drivers behind the development of middleware was mergers and acquisitions. When companies merged, they found they had information stored in separate systems and locations but no way to share it."
Consequently, companies began to write their own code to connect these systems--a practice often called "rolling your own"--and middleware was born. The need for system integration increased with the advent of multifaceted ERP suites and more advanced enterprise technologies. Though IBM is given credit for creating the first standardized middleware solution, by the early '90s, Big Blue had considerable competition as middleware became a standard requirement for enterprise software implementations.
Who makes middleware? While there are literally hundreds of software companies offering integration technology in one form or another, in the middleware industry there are
generally gorillas and pure-plays. The gorillas are giant software companies such as IBM, Oracle, Microsoft, Sun, Sybase and Computer Associates (see "Middleware Vendors," this article). These behemoths typically offer integration technology as part of an e-business platform. For instance, IBM--the original middleware vendor--offers WebSphere, an e-business app-server platform that features a middleware layer designed to interface with leading business applications. Its competitors in this market follow suit, with platforms featuring sophisticated integration functionality. According to IDC's Sally Hudson, the app-server model is the wave of the future. "For designing and deploying new applications, the way to go is the app-server platform. These platforms include the ability to tie in legacy applications as well."
The pure-plays focus solely on middleware, which some argue gives their products a technological edge over their larger, more diversified competitors. Pure-plays currently moving and shaking in the middleware industry are, among others, Tibco, Vitria, DataMirror, SeeBeyond, webMethods and Propelis. While each of these companies offers integration solutions, they differ in technology used and application focus. For instance, DataMirror focuses on real-time enterprise data delivery, an area integral to the success of CRM initiatives.
Since its birth in the 1970s, the middleware industry has grown in proportion to the growth of enterprise applications. IDC reported that in 2000, the middleware industry generated $3.4 billion in revenue, a figure expected to reach
$6 billion by 2005. This reflects a compound annual growth rate of 12 percent per year between 2000 and 2005.
How much does it cost? As with most business software,
the cost of middleware depends on the number of users, the amount of customization required, who exactly is selling to whom, and a host of other factors. Therefore, it is impossible to provide a standard price. However, Tyler McDaniel of Hurwitz Group estimates that large firms pay somewhere in the neighborhood of $500,000 to middleware vendors for an integration platform. Once again, this number applies only to large firms with large needs.
In the absence of hard numbers, it may be helpful to
consider the cost of achieving your business/technological goals without middleware. On this topic, it comes as no surprise that middleware vendors are quick to point out just how expensive and inefficient it is to roll your own. "To write a single interface between ERP and CRM takes two weeks of design time, four weeks of coding, two weeks to test and another week to implement," says DataMirrors' stokes. "Coders make $800 per day, and for nine or 10 weeks, that comes out to roughly $36,000. At that cost, middleware would pay for itself in one implementation."
While middleware vendors have an obvious interest in downplaying the roll-your-own option, the fact remains that implementations of CRM and other enterprise solutions are difficult and costly. Because vendor middleware is designed to interface with leading CRM applications, and because it is almost always upgraded regularly, it allows users to bypass a significant amount of implementation-related pain. "Middleware is an important part of almost any CRM implementation," stokes adds. "Up to 60 percent of implementation costs can go to interfacing. Unfortunately, that cost is often underestimated. People say, ‘we'll just get the information from our mainframe.' That's easy to say, but it is not easy to do."
I'm planning a CRM implementation. What do I need to do about middleware? If CRM is an add-on to an already fully integrated enterprise system, then you probably won't have to do much. "Most CRM vendors have an imbedded middleware product," Boucher says. "If you are going to buy CRM, you should ask what type of middleware the vendor uses, and then confirm that it will work within your system."
However, if you are coming late to the enterprise integration game and you still have '80s-era technology silos, you should work with an integrator and your implementation team to design and create a solid integration strategy. In it you will identify all touch points between the new CRM system and existing applications. Then search for a middleware vendor with expertise in your vertical, and with solutions that will allow your system to meet your present and future needs.
Above all else, an integration strategy should reflect your
company's overall business strategy. "An integration system is not a solution," McDaniel says. "As a business decision maker, you should focus on what you need to accomplish, and how certain applications will help you accomplish it. In all likelihood, an integration implementation will enhance that strategy."