• August 7, 2020
  • By Paul Greenberg, founder and managing principal, The 56 Group

With Big Data, Big Isn’t Necessarily Better

Article Featured Image

I usually try not to write about technology even though I’ve dealt with it for more than two decades. But I keep seeing the same issues come up repeatedly—and they seem to be continually escaping the notice of all but the most accomplished practitioners, so I feel compelled to write about them.

First, technology isn’t a panacea, it’s an enabler—though at times it can even be a driver.

Second, selecting your technology is the last thing you do, after you’ve determined your strategy and programs, designed your business processes and workflows, created your business rules, and identified the specific needs and requirements you have for the technology—and not until then.

If you have a growing company and are serious about engaging with your customers, you are going to have to use some technology. The question becomes not whether to use technology but how to decide which technology to use.

Which leads to the third thing: If the data you capture is left alone, it’s pretty useless, but apply the right technology and it becomes your company’s lifeline.

As I hope most of you know, capturing and organizing data is the No. 1 reason to think about using technology. When it comes to your mission-critical initiatives around customer engagement and customer experience, well, that’s where the use of data is paramount. So let’s talk data.


One thing we can be sure of: We need to know more about our individual customers, and not just what slot they fit in a demographic analysis.

Immediately following the head nodding that accompanies that statement, you’ll often hear, “Well, this is the era of Big Data and therefore…” And from there, you’ll hear some stats related to petabytes or zettabytes and some homily to the amount of data available. And if you hear all that, it means someone, and I won’t mention names, isn’t getting it.

It isn’t the amount but the specific kinds of data that matter, as well as the insights you gain. In recent years, we’ve been able to gain prodigious amounts of industrial-­strength transactional and demographic data about individual customers. But what makes this era unique and incredibly exciting is that we now have gobs (a scientific term for huge amounts) of behavioral data that, together with the transactional and demographic data, provides us with detailed profiles of customers and how they’re acting at any given time. When analytics programs do what they do, the data yields insights into individual customers in real time and allows responses to them based on present or anticipated behavior.

Unfortunately, marketers are often behind the curve in understanding who customers are, what they want, and what they’re doing, relying for personalization too much on demographics and transactional data like email addresses and not enough on location, lifestyle, and psychographic data. While marketers pay homage to customer engagement, which requires their knowledge of behavior, they don’t gather the information they need to make appropriate judgments.

Luckily, more companies are trying to engage with customers in immersive ways, which means, of course, getting to “know” individual customers and how they act and what they like. When you’re trying to personalize customer relationships, having behavioral data is where you start to distinguish yourself. The challenges, however, are significant.


The obstacles technology is designed to overcome are non-trivial: scale, volume, velocity, variability, veracity, and asking the right questions. Let’s break them down.

Scale: Capital One (my bank) has 45 million customers; Amazon (my preferred shopping) has 300 million customers; Verizon Wireless (my mobile carrier) has 146 million customers.

In total, I’m three of 491 million customers. Yet I don’t care what the other 490,999,997 customers want from the companies. What I want from each is specific.

• Some explanation as to why I can’t get specific information about my account at Capital One without having conniptions—I feel insignificant.

• Why the product that Amazon shipped second-day air didn’t arrive on time. In response, I get an explanation and a free month of Amazon Prime for my troubles—I feel important.

• Why I continue to get offers that are unrelated to me or my status with Verizon Wireless—I feel misunderstood.

Each of them could accommodate me with the right data about not just my transactions but my interactions and other behavior. The dilemma: how to do this for their entire customer base. It isn’t easy providing what each customer wants: personalized interactions that scream: “HEY, WE KNOW YOU—AND WE CARE,” or at least say that in lowercase. But technology allows for the identification, capture, and storage of relevant individual customer data, actions to take on that data, and the organizing of those actions into usable form (i.e., transitioning data into information). When employees glean insights from the organized reporting of data, the information becomes knowledge, and that leads to applied insights. The industry cliché—for those who use industry clichés to show how smart they are—is “actionable insight.”

Volume/velocity: Many articles and posts expound on the 4.4 zettabytes of data (as of 2013) that is projected to grow to 180 zettabytes by 2025 (IDC’s forecast); if we just eliminated the articles, we’d probably reduce the total data to 2.2 zettabytes. What’s more interesting and far more daunting is the velocity of the data—the continual increase in the rate of data production, year over year. That, in combination with the total amount, is what has been dubbed “Big Data.”

Technology, it was said, couldn’t handle the volume or the velocity of Big Data. Well, that was nonsense. Technology was designed and developed to handle it. Now the germane question—a real one—is: How do we gain insight from that data? The phrase often used is “Big Data, small insights.”

Variability: The kinds and sources of data vary widely—transactional data, social data, sentiment data, mobile data, sensors data, video data, audio data, geospatial data, data from apps usage. There’s more, but you get the point. While many queries still target the structured data from internal systems, the need for individual customer information and the response time available to answer customer queries make the value of other data sources immense. The short window of opportunity to respond is also why having enough customer journey data to anticipate customers’ behavior using predictive and, eventually, prescriptive analytics is increasingly appealing.

Veracity/clarity: In the past, with the structured transactional data that traditional CRM storehouses held, your concern was along the lines of: “Brad Pitt, Bradley Pitt, Brad Pit—all the same person?” Then as social media use became widespread, the concern was “Is Brad Pitt the name and address in Los Angeles the same as bradpitt on LinkedIn, @bradpittoceans11 and @bradpittoceans12 on Twitter, and facebook/bradpittaloneagain on Facebook?” Tech companies like Gigya make identity verification a lucrative business. At last count in 2018, Gigya had 1.2 billion verified social IDs. But the amount and variety of available sources and data types that need to be verified and ready to be analyzed have gone well beyond just name and address ID verification and social ID verification. Now clarity matters as much as veracity. You have to deal with misspelled text in tweets, or the context of social media conversations and sentiments, not to mention jargon, multiple languages, different cultural contexts, abbreviations. Clarity and veracity are complex challenges that can, if improperly done or ignored, impact results so severely that you will make the wrong decisions or respond the wrong way to individual customers and damage rather than improve the relationship. But the right way? The benefits can be significant.

The right questions: One theme you will see everywhere when it comes to data, analytics, artificial intelligence, and machine learning is that you should ask the right questions to get the right answers. You can ask the wrong question and get the right answer to the wrong question, which is, for your actual purposes, the wrong answer. But it will seem right. Even machine learning can be dumb: It needs to be pointed in the right direction to learn the right answers. Asking the right questions, knowing what you are seeking to find, is vital to the successful use of data for insight.

In other words, “Big Data, small insights” potentially leads to happy customers, and maybe, just maybe, loyal customers, if you can meet the challenges.

How you capture, store, report, analyze, and interpret data and do so in a way that gives you and your customers what you want—well, that’s what systems of record and systems of engagement are there for.

These aren’t challenges germane to just a small sampling of companies or industries. Institutions across the planet are hit with the same questions that need to be answered. In a time when uncertainty about pretty much everything reigns, data and the ability to interpret and having the technological firepower to do so becomes super-important to the well-being of companies and even individuals. Who wouldn’t want that? 

Paul Greenberg is the author of CRM at the Speed of Light, called the “Bible of CRM.” His latest book is The Commonwealth of Self Interest: Business Success Through Customer Engagement (2019), available on Amazon.

CRM Covers
for qualified subscribers
Subscribe Now Current Issue Past Issues