The Dangers of Out-of-Date Data

Can there be such a thing as a wrong time to use right-time data? The answer is yes -- especially when real-time data is concerned. Real-time data infiltrates into business applications, giving users the most current, up-to-date information possible. Out-of-date data is so yesterday. Yet even with all the perks that real-time data provides, a new report from Forrester Research indicates that there are limitations on when “real-time” data should be used -- and when “right-time” data will suffice.

In the report, "Really Urgent Analytics: The Sweet Spot for Real-Time Data Warehousing,” Forrester analyst James Kobielus dives into the state of real-time data warehousing and business intelligence (BI). “One misconception is that [handling real-time data is] more complex than standard data warehousing,” says Kobielus, who is also a columnist for CRM magazine. “Real time can be done with the exact same tooling and infrastructure that you use on existing data warehousing and BI.” He says that he sees information technology and knowledge management professionals increasingly looking to real-time approaches to leverage and optimize customer data. 

He recommends that an organization take a good look at its data priorities before really jumping into real time. “Number one is to determine for your existing environment where you need to go real time versus what will be well served by doing overnight batching,” Kobielus says. “Don’t overstate the amount of real-time access that customers require.” He adds that continuous refreshing can in some cases devote unnecessary disk space and take away from other core business applications. “The number one challenge for BI and data warehousing professionals is to determine what percent should be real time and what should be right time,” he says. In addition to choosing which business processes are most suitable for a real-time approach, decision-makers must also choose the most suitable path. Kobielus' report lists the following types of real-time approaches to data warehousing -- a list that he cautions is not exhaustive. They overlap and are not mutually exclusive.

• Real-time data warehousing (RDW): Trickle feeds or incremental batches refresh data.
• Operational data store (ODS): Offloads current or near-real-time reporting from applications for faster implementation than RDW.
• Event-stream processing (ESP): Subsecond end-to-end latency from the source to the consuming application is guaranteed.
• Data federation: Uses a semantic layer to update heterogeneous data.
• Information fabric: A caching infrastructure is embedded in services-oriented architecture or event-stream processing for analytic and transactional applications.

Kobielus says he currently sees event-stream processing gaining momentum out of the real-time data warehousing segments. He adds that down the road -- in about 10 years, by his estimate -- information fabric will most likely be the dominant pattern. In general terms, Kobielus suggests that we are seeing more and more real-time approaches to data as enterprises look to “true” real-time application implementations.

“True” real time, he says, refers to the continuous streaming of data into applications -- data that refreshes with guaranteed delivery. A robust middleware assures that, once a value is updated, the change is immediately pushed into other applications. More and more BI applications are relying on that middleware to provide the framework for such real-time actions. 

News relevant to the customer relationship management industry is posted several times a day on destinationCRM.com, in addition to the news section Insight that appears every month in the pages of CRM magazine. You may leave a public comment regarding this article by clicking on "Comments" at the top; to contact the editors, please email editor@destinationCRM.com.

Related Articles

Still No Masters in Gartner's 'MDM for Product Data' Magic Quadrant

The latest assessment of master data management for product data covers 11 vendors -- and reveals no leaders. Is product data that much harder than customer data?

Self-Service Business Intelligence Is a Myth

TDWI Spring '08: Line-of-business users want access, but self-service is only a short-term solution.

The Stewards of Business Intelligence

TDWI Spring '08: The most important element in BI -- the human factor -- is also the most-often overlooked.

Dirty Little Data Secrets

CRM's real truth requires enterprises to clean up -- reorganize -- customer information with data integration solutions, Web-services technology that integrates data applications.

BI Tools to Manage Real-Time Response

Additions to Business Objects' operational BI platform still require integration, but address the trend of operational and tactical analytics.

Serving Many Masters

Customer data and business data must be controlled through master data management.

Mastering Customer Records

How services-oriented architecture will shape the future of CRM: Part One in a four-part series.