Mastering Data Requires Attention to Detail

The inconsistencies exist because fixing such problems is a monumental task, one that requires companies to tackle both technical and organizational issues. Master data management (MDM) solutions, which have been sold for decades, are designed to address the technical issues. They are built to clean up the various inconsistencies, a process dubbed data cleansing.

The work sounds straightforward, but it is time-consuming and excruciatingly complex. The business has to audit all of its applications and determine what is stored where and how it is formatted. In many cases, businesses work with terabytes and petabytes of information. Usually, they find many more sources than initially anticipated because cloud and other recent changes enable departments to set up their own data lakes.

STARTING THE CLEANSING PROCESS

Cleansing starts with mundane tasks, like identifying and fixing typos. The MDM solution might also identify where necessary information is missing.

To start the process, companies need to normalize fields and field values and develop standard naming conventions.

Such work is time-consuming, and with good reason: While the computer industry has developed standards in many areas, none exist for MDM. “I am skeptical of data formatting standards because each business uses information in unique ways,” says Eric Melcher, vice president of product management at Profisee, an MDM platform provider. He also points out that the sheer volume of data, the new ways that companies find to use it, and the complexity of business applications make moving targets out of any potential standards.

However, the data clean-up process can be streamlined in a few ways. If a company chooses only one vendor to supply all of its applications, the chances of data having a more consistent format increase. Typically, vendors use the same formats for all of their solutions. In some cases, they include add-on modules to help customers harmonize their data.

But that is not typically the case. Most companies purchase software from different suppliers, and data cleaning has largely been done in an ad hoc fashion, with companies harmonizing information application by application. Recognizing the need for better integration, suppliers sometimes include MDM links to popular systems, like Salesforce Sales Cloud, Microsoft Dynamics, and Marketo. The chances of finding links to other relevant applications, such as ERP, product life cycle management, and billing systems, are remote.

The work is also labor-intensive, and many MDM projects can take years to finish. “There is a heightened need to curate (classify, tag, audit, etc.) data at scale as our information ecosystems grow and federate,” says Forrester’s Goetz. “Manual efforts are inadequate.”

Artificial intelligence and machine learning are emerging to help companies grapple with such issues, but the work is still in the very early stages of development.

Still other challenges stem from internal company policies—or a lack thereof—and corporate politics. Businesses need to step back from their traditional departmental views of data and create an enterprise-wide architecture. They must understand data hierarchies and dependencies; develop a data governance policy; ensure that all departments understand and follow that policy; and assign data stewards to promote it.

But such organizational change often encounters problems right at the start. The simple question of who is responsible for cleaning the data often yields a convoluted answer.

With the advent of cloud technology several years ago, many business units gained autonomy over their departmental applications. The relationship between these groups and IT has sometimes been strained. The latter’s objectives to keep infrastructure costs low and to put central policies in place to create data consistency often conflict with the business unit’s drivers, which largely center on responding to customer demands ASAP. And while departments have taken more control over the data, they often lack the technical skills to manage it on their own.

A FAILED LEGACY

Resistance to MDM has been common for a number of other reasons. For starters, departments often lack the budget and sometimes the interest to commit needed resources to MDM projects, according to Gartner’s O’Kane.

And because the technology and organizational barriers have been so high, many MDM projects have failed. “MDM has a negative connotation because it is viewed like a typical IT project—long and expensive,” Openprise’s Pogorzelski adds