Solving the SOA Data Dilemma
A recent study by analyst firm IDC estimates that the average company has 49 applications in 14 different databases, which need to be integrated, and typically has no more than 20 percent of its customer data residing in any one location. IDC also reports that over the next three years the world's data will increase six fold annually. At most companies, the number of systems continues to grow, not shrink, which is exacerbating the problem of data proliferation.
Most business leaders agree that data is a critical strategic asset, yet effective management of information has remained elusive. The core of the problem is an inability to easily share data between systems or make systems work better together. By properly employing a services-oriented architecture (SOA), enterprises can leverage their existing systems, while largely leaving them alone, and create a new integration solution for more effective information sharing across disparate applications.
There are two basic kinds of SOA services: process-centric and data-centric. Process-centric services execute business processes such as authorizing credit cards, processing orders, sending bill, etc. Data-centric services manage the attributes and relationships of data required by process services. Both services can be served by a data-centric platform that is responsible for only one thing: managing, in a single place, the uniqueness, integrity and interrelationships of business data that matter most. This is what a Master Data Service (MDS) provides.
Within the platform, each MDS is the authoritative source for a particular type of "master" data (customer, product, license, location, event, asset, and portfolio, to name a few). Master data are those that are generally the most highly shared and the most critical to successfully meeting the goals of an enterprise.
An MDS provides an ideal way for managing data within an SOA environment. Using a "hub and spoke" model, the MDS serves as the integration method to communicate between all systems that produce or consume master data. The MDS is the hub, and all systems communicate directly with it using SOA principles (XML documents exchanged over HTTP or JMS).
While MDS deployments solve many of the common issues of enterprise-wide data management, the following guidelines should be followed to ensure success:
Manage Data Quality Rules in One Place -- An MDS provides the central tooling for validating the completeness and accuracy of the master data it manages. It should apply one consistent set of policies and rules, so that the best information is available to all participating systems. For example, there should be one service to validate postal addresses, ensure all systems are using the same product description, or provide correct product pricing to multiple order-taking applications.
Control Data Redundancy -- Data housed in multiple applications and databases often contain similar records about customers and transactions. An MDS should be the single place for managing the uniqueness of master data records across all participating systems, even though data are duplicated across (and sometimes within) systems. Rules in the MDS should help resolve identity questions and where and how to obtain the most accurate information. For example, the MDS might use the ERP application for billing addresses and the CRM application for shipping addresses.
Resolve Semantic and Structural Variations -- Every application uses a unique schema for managing its own data -- that's completely normal. However, in order to share data among disparate systems, it is critical to implement a system for mediating these differences. An MDS should hide semantic and schematic differences and create a consistent standard between applications.
Require Autonomy -- Each system that produces or consumes master data should only have to "talk to" a single MDS for each kind of master data. This relieves each system from having to know the details about other master systems, or share the same set of business rules for managing overall data quality. It also greatly simplifies the process of adding new systems, or sun setting systems -- MDS makes this an addition problem instead of a multiplication problem.
Enforce Data Governance Policies -- Data governance is the set of policies, procedures, processes, skills and technologies required to manage data as a strategic asset. It is 80 percent business processes and 20 percent technology. Establishing and resolving data quality and business rule control issues between policy administrators of each system or business unit is a difficult process. An MDS solution helps resolve these tasks by centralizing and streamlining all of the policy administration for capturing new data and enforcing data quality policies.
Delivering an enterprise-wide, authoritative source of master data is the role of an MDS. It understands all of the characteristics of enterprise-wide data, matches and links data accurately and automatically between all internal and external data services, and enables the non-invasive, non-disruptive delivery of information to existing business systems via an SOA.
About the Author
Marty Moseley serves as chief technology officer at Initiate Systems (www.initiatesystems.com) where he is responsible for the company's strategic technology direction, development and future product evolution. Initiate Systems enables organizations to strategically leverage and share critical data assets. He can be reached at mmoseley@initiatesystems.com.