Your AI Agent Read the List, Not the Letters
Backstory CEO Jason Ambrose recently argued on this site that agents don't care about the interfaces we built for humans. They take whatever signal is cheapest to reach. A health score is cheaper than the account history behind it. A subject line is cheaper than the message it summarizes. I ran into the same pattern while building an email and calendar integration for a revenue team's morning pipeline review. The answers sounded informed. Many of them weren't.
The failure showed up in an evaluation, not a demo. I asked an agent to triage a 48-hour window of a sales inbox. It inventoried 125 emails and opened the bodies of 17. Body-read coverage was 13.6 percent, and no attachment was opened. Most of the remaining messages, roughly 108, were still sorted into categories using only subject lines and snippets. The output looked complete: every message had a label, and the summary read well. A sales leader skimming it would have had no reason to doubt it.
Two things went wrong. Neither is exotic.
First, the agent stopped scanning while there were still records to fetch. It read a "maximum results" setting as "stop after this many," when it was meant as "scan everything, then return the top items." Second, nothing in the system distinguished a message that had been read from one that had only been listed. Both produced the same kind of label.
In a separate test reconstructing a negotiation thread, the agent's summary was commercially sensible, yet parts of it described messages it had never opened. The answer was useful, but its claim to be based on evidence was false.
Customer-facing teams should care about that difference. A summary built on metadata can be right most of the time. When it's wrong, it's wrong about the things that hide in bodies and attachments: a changed renewal date, a procurement condition, a pricing objection, a contract redline. Those are exactly the details a CRM agent is supposed to surface.
The fix wasn't a better prompt. Telling a model to "read carefully" doesn't hold up across thousands of runs. The rules have to live in the interface the agent works through. Here are three changes any team running agents over CRM and communication data can adopt, whatever tools they use.
- Separate discovery from evidence. A listing tool should return an inventory: sender, date, subject, thread, whether an attachment exists. It shouldn't pretend to be a reader. A separate tool should do the evidence read and return the full normalized text, page by page, along with a flag saying whether more text remains. The agent should be told plainly, inside the tool's own description and not only in a system prompt, that it cannot categorize, extract amounts, or assign follow-ups from a listing alone.
- Make "unread" a first-class state. If a record wasn't opened, the output should say so with an explicit category, something like "unread, metadata only," instead of a guessed label. This one change turns a silent gap into a visible one. A manager reading the brief can see that 108 of 125 items were never opened and decide whether that's acceptable, which is a decision the agent shouldn't make on its own.
- Report coverage alongside the answer. Every summary an agent produces from records should carry a short coverage line: how many records were found, how many were read in full, whether the scan reached the end of the results, and which attachments were opened. That line costs almost nothing to produce and gives reviewers, evaluators, and auditors something to test. It also gives you a simple acceptance rule. For a pipeline review, you might require that every message assigned an action was read in full, and treat anything less as a failed run rather than a slightly worse answer.
There's also an organizational lesson. In my first design, one "parent" agent did everything: scanning the inbox, rebuilding threads, triaging attachments, reconciling with the CRM, and writing the final brief. Under that load, it took shortcuts. Splitting the work helped. A narrowly scoped worker was told to be exhaustive and to return structured findings with coverage and gaps. The parent agent then judged and wrote. The worker owns completeness, and the parent owns judgment. That division mirrors how good sales operations teams already work: the person who compiles the account history isn't always the person who decides what to do with it.
None of this requires a particular vendor, model, or protocol. It applies to agents connected to email, call notes, support tickets, or CRM objects through any integration layer. It requires a change in how we evaluate agents. Most teams grade the final answer: was it accurate, useful, well written? That's necessary but not sufficient. An agent that reaches a good answer from evidence it never read has gotten lucky. Luck doesn't scale, and it can't be audited.
Before you let an agent write to your CRM, prioritize your pipeline, or brief an account executive, ask one question of every output: what did it actually read? If the system can't answer, that's the first thing to fix.
Tool second. Evidence first.
Fedor Kovalev is founder of Cheetah, where he leads go-to-market strategy, systems work, and delivery. He was the sole builder of a Slack-native, multi-agent revenue platform that used custom MCP servers, evaluations, and human approval gates for external sends and writes. Earlier he was sales development lead at ElectroNeek and programmatic sales director at Hearst Shkulev Media.