Tips for Dodging AI Disasters

We’re not even one full quarter into 2024 and it feels like every week I’m seeing new stories outlining very public customer service whoopsies as companies find their footing with new artificial intelligence deployments.

Does this surprise me? No, definitely not. We’ve seen truly enormous leaps in AI sophistication in the past year or so, and it’ll take time to work through that learning curve. But it does compel me to give you a couple tips on how to avoid missteps in your customer service AI strategy:

Fancy Algorithms Can’t Save You From Inconsistent Knowledge

You might have seen a major airline get into some legal hot water recently due to its chatbot giving a customer incorrect information. If you read between the headlines, though, you’ll notice that it had literally nothing to do with generative AI hallucination and everything to do with internal knowledge inconsistencies.

These days, it’s easy to assume that these stories are a result of chatbots gone rogue, but the issue is rarely the choice in algorithm; instead, it’s often the strategy (or lack thereof) for ensuring that these solutions are grounded in accurate enterprise knowledge.

Whether you leverage traditional natural language processing techniques or generative AI, your source of truth needs to be, well, true.

If you haven’t already invested in a strong knowledge strategy (and I’m not just talking tech—internal processes matter too), this is going to be your best first step. Any subsequent investments in customer service AI, be it customer-facing chatbots, agent-guidance tools, or those fancy generative AI co-pilots, should be firmly grounded on a rock-solid knowledge foundation.

Generative AI Is Not Right for Every Use Case

Don’t shoot the messenger, but it’s true!

I’ll give you an example: Could you use generative AI to develop a solution for auto-scoring your contact center conversations? Sure. But should you? Not necessarily.

Your “traditional” quality monitoring approach will have a human quality analyst manually scoring around 2 percent of an agent’s calls each month. We can all agree that that’s a pretty bad ratio. Plus, manual call listening is not where our human talents shine brightest. Automated quality monitoring solutions are a bit of a no-brainer; let the machine listen to 100 percent of your calls and auto-score the obvious stuff, and have the human analyst handle a subset of scores that require more nuance and consideration.

Now some might say: “Hey, Christina, why would I build out a whole taxonomy when I can just have genAI score my calls for me?”

Short answer? It would be friggin’ expensive.

Every time you ask a genAI model anything, the act of answering costs something. If you’re a contact center handling 500,000 conversations a month, running each conversation through a generative model would cost you several somethings—at least with today’s math. Sure, you could reduce the percentage of calls you send through the model, but then we’re back where we started with subpar sampling.

Generative models are non-deterministic. This means that if you ask the same question five times, you are unlikely to get the exact same response. There are ways to mitigate these inconsistencies, but it’s not all that simple. In situations like quality monitoring, where you need things to stay consistent and fair, trying to make a genAI model act like a traditional language model just doesn’t add up, especially when it comes with a much higher price tag.

In some cases, generative AI is just way too big of a hammer for way too small of a nail. In others, it’s like using a sledgehammer to knead focaccia.

Look, I never said I was the master of metaphors. Bottom line: Pick the right tool for the job!

Christina McAllister is senior analyst, Forrester Research, covering customer service and contact center technology, strategy, and operations.

Related Articles

Does Contact Center Tech Translate to Physical Stores?

Post-pandemic, the digital and physical realms are merging,

Manifesting Better Customer Experiences

Three things I predict (hope?) will happen this year.

Three Contact Center Resolutions for 2024

New year, new queue! Here's how to make 2024 better for customers and agents alike.

An Insider’s Guide to Navigating Customer Service

Some tips and tricks I've learned, both through personal and professional experience.