What Needs to Be in Place Before an Agent Answers the Customer
A customer asks why a charge appears twice. The agent answers in less than a second, in complete sentences, with the tone of someone who has handled the question 1,000 times. There is one problem: It read last quarter's billing rules and answered for a policy that no longer exists. The customer acts on it. By the time anyone notices, the wrong answer has a name attached to it: yours.
I build production AI agents for advertiser analytics at Amazon Advertising. My agents answer questions from advertisers and their account teams about campaign performance and audience data. The questions are commercial, not consumer support, but the failure is the same one CX teams are meeting now. An agent gives a fluent, confident answer built on the wrong record, and nothing in the wording gives it away. Fluency is not evidence. Format is not evidence. The only evidence worth having is what the answer read, which rules governed it, and whether anyone tested it against known right answers before it shipped.
The audience strategy agent on which I work runs 24 governed playbooks, one for each recurring job it does. A playbook is a fixed set of steps: what the agent reads, which rules apply, and what a finished answer is allowed to contain. When a customer's question falls outside the playbooks, the agent does not improvise. It hands the question to a person.
Every playbook carries an evaluation set, a bank of questions with answers we have already checked by hand. Any change to the model, the instructions, or the tools runs the full set before release, and a failed evaluation holds the release. Not a warning in a dashboard. The release waits until the answers are right again.
I designed and proof-of-concepted the orchestrator that routes questions to the right playbook, and the part of that work people underestimate is the routing itself. The agent has to pick the right procedure before it answers, the same way a good service rep opens the right policy page before speaking. Most wrong answers I have reviewed were retrieval failures wearing a confident voice, not reasoning failures.
Vendor-neutral, this is a discipline any team can copy. Write the procedure down before the agent answers from it. Test every change against answers you already trust. Treat a failed test as a stop, in the same way a failed reconciliation stops a close.

Figure 1. Three gates an answer passes before it reaches the customer: a governed playbook, an evaluation gate, and written scope over the current record. Questions outside the playbooks hand off to a person.
One repair from my own work explains more about agent quality than any model upgrade I have seen. A metadata lookup behind our answers scanned an entire table on every call and took about 3.7 seconds. Worse, the agents worked from whichever record the scan happened to reach. We replaced the scan with a targeted query. The lookup fell to about 200 milliseconds, and every answer after that was tied to the exact record it claimed to describe.
Customer-facing teams live with the same problem in a different costume. The CRM record holds three versions of the account. The knowledge base holds last year's policy next to this year's. An agent reading from that pile will answer fast and wrong, and it will sound wonderful doing it. Freshness is not a nice property for an agent's data. It is the difference between an answer and a rumor. Before an agent answers a customer, someone should be able to say which record it read and when that record was last true.
Every agent run costs money. It reads data, calls tools, and spends compute, and an agent with wide access spends on everything it can reach. I watched one internal workflow, scoped more broadly than the jobs around it, cost several times what they cost. Nobody caught it in review because its outputs looked fine. Scope creep inside an agent does not announce itself as spend. It announces itself as slightly wrong answers nobody traces back.
Written scope fixes both problems at once. Each playbook in my work reads the accounts and date ranges its job needs, nothing wider, and the spend follows the job instead of the data lake. The same discipline serves a customer-facing agent: the billing agent reads billing records, the delivery agent reads delivery records, and neither one browses the rest of the customer file because it can.
When a platform offers your team an agent for customer questions, the demo will be fluent. They always are. Ask three questions instead. Which procedures does the agent follow, and who wrote them? What tests run before an update reaches my customers, and does a failure stop the release? Which records can it read, and can you show me? A vendor who answers all three plainly is selling you a governed system. A vendor who answers with fluency is selling you a voice.
Customers will not grade your agent on its tone. They will grade it the way they grade every employee who ever answered them: on whether the answer was right and on what your company did when it was not. The playbooks, the tests, and the scope are how an agent earns the right to speak for you.