The way to summarise customer feedback is to structure it first and summarise second. Every comment broken into mentions, each mention given a topic and a sentiment score by one consistent method, all of it stored. A summary written on top of that points you at what matters, ranked by impact rather than volume, and you can open any line and read the comments behind it. A summary written straight from raw text is a plausible paragraph you cannot check.
Key takeaways
- A summary is a headline, not the story. It tells you what to read next
- Summaries let you down in two ways. They invent something the comments never said, or they quietly leave out something they did. The second is worse, because nothing looks wrong
- Manual summarising fails at scale because the summariser picks the loudest comments, not the most important ones
- Summarise structured data, not raw text, and the summary inherits the counts, the scope and the topics
- The summary tells the analyst where to look. The analyst drills down to the verbatims and decides
Someone has to tell the leadership team what 8,000 customers said this quarter, in four bullet points, by Thursday.
For years that person was an analyst with a highlighter. Read as many as you can, pull out the ones that jump out, write it up. Everyone knew it was a sample. Everyone pretended it wasn’t.
Now the job often goes to a language model, or to a VoC platform, and the temptation is to paste the 8,000 in and take what comes back. That is faster. Done badly, it has the same problem, plus one new one.
A summary is a headline
A headline is useful because it leaves things out. Eight words tell you whether the story matters and whether it’s yours. Nobody mistakes the headline for the story. You read it, decide it matters, and read on.
That is what a feedback summary is for. It shows you where to focus. Complaints about the returns process are up, praise for the new app is holding, one region is an outlier. Now you know where to spend the morning. The morning itself is spent in the comments.
Treat the summary as the story, and the moment someone asks a follow-up question, there is nothing underneath.
The two ways a summary lets you down
The first is loud. The summary says something the comments never said. It invents a theme, a cause, a number. ‘Customers are frustrated by delivery costs’ when nobody mentioned cost. A careful reader will sometimes catch it.
The second is quiet, and it is the dangerous one. The summary leaves something out and still reads clean. A whole topic vanishes. A qualifier drops. ‘Customers found onboarding confusing’ is a finding. ‘Customers on the old tariff found onboarding confusing after the March change’ is a decision you can act on. The first isn’t wrong. It has thrown away the part that made it useful, and no reader can tell.
Both get worse with volume. Research on language models finds that the more text you hand a model at once, the more it invents, and a summary that condenses 8,000 comments into four lines is throwing away 7,996 comments’ worth of detail by design. The question is whether it threw away the right ones.
Why summarising by hand stops working
An analyst summarising 300 comments does a decent job. The same analyst summarising 8,000 does not, and it is not their fault.
Attention runs out. The loud comments win, the angry ones, the funny ones, the ones near the top of the spreadsheet. The quiet pattern spread thinly across 400 mildly disappointed people never makes the cut, and that pattern is often the one costing you customers.
Categories drift. What counted as a ‘service’ complaint on Monday morning has become three different things by Friday. And nothing is stored. Next quarter’s summary is written from scratch, so nobody can say whether returns complaints went up or down. They can only say what stood out this time.
Pasting the same 8,000 into a general model fixes the attention problem and keeps every other one. Which comments got weighted. Which got skipped. Whether the counts in the paragraph are counts at all. Nothing is stored, so next quarter is another fresh reading. Why that matters is the whole of where ChatGPT works and where it breaks.
Summarise the structure, not the text
Here is the change that fixes most of it. Don’t summarise the comments. Summarise the data the comments have been turned into.
Structure first. Every response broken into mentions. Every mention filed under one topic and scored for sentiment, using the same method for surveys, reviews, tickets and social. All of it stored, with the context of who said it, where and on which channel.
Now a summary is written on top of something real. When it says returns complaints are up, that is a count of mentions with that topic, this period against last. When it ranks what to worry about, it ranks by impact, how many people said it combined with how strongly they felt, so the quiet pattern across 400 people outranks the one furious paragraph. And when it says ‘customers on the old tariff’, that scope came from the data attached to the responses, not from the model deciding what to keep.
A summary built this way can still miss things. It is a summary. But it cannot invent a topic that isn’t in the data, because the topics were assigned before the summary was written, and the numbers it uses are counts rather than impressions. SmartCX gives every team one trusted data foundation for every dashboard, decision and action, and the summary is one of the things built on it.
Scope before you summarise
A summary of everything is a summary of nothing. Narrow first.
One journey stage. One region. One product line. One quarter against the last. The smaller and sharper the question, the more the summary can say something specific, and the more the qualifiers survive. ‘What are people saying?’ gets you a shrug in prose. ‘What changed for first-time buyers at checkout since the release?’ gets you a finding.
Scoping also keeps the summary honest. If the answer is ‘not much’, a well-scoped summary of structured data can say so. A general model summarising everything will find something to say regardless, because that is what it is for.
Then drill down
The summary has done its job when you know where to look. What happens next is the point of the exercise.
Open the topic. Read the comments underneath, with the sentiment on each phrase. The summary said returns complaints are up. The comments say it is the pre-paid label that never arrives, not the returns policy. That is the fix, and it was never going to be in the four bullet points.
This is what SmartCX’s summaries are for. They show you where to focus, so your team spends less time hunting for what to act on and more time on the thing itself. The analyst still verifies, still investigates, still makes the recommendation. Drilling down to the verbatims is how they do it, and how to do that properly before a finding goes anywhere is in how to validate AI insights.
Four bullet points by Thursday is fine. Just make sure there are 8,000 comments underneath them, filed, scored and one click away.
That’s how SmartCX is built. Feedback structured into stored, scored mentions by one consistent method, summaries that point at where to focus, and the comments behind any line one click away. See it on real feedback in our product tour.
Frequently asked questions
How do you summarise customer feedback?
Structure it first. Break every comment into mentions, assign each mention a topic and a sentiment score using one consistent method, and store the results. Then summarise the structured data, scoped to a specific question, and treat the summary as a pointer to where to read the underlying comments rather than as the finding itself.
Why do AI summaries of feedback go wrong?
Two ways. They invent something the comments never said, or they leave out a topic or a qualifier and still read clean. Both get worse as the volume of text handed to the model grows. Summarising structured data rather than raw text stops the first and reduces the second, because topics and counts exist before the summary is written.
Can you trust an AI summary of customer feedback?
As a guide to where to look, yes. As a finding to act on, only after someone has drilled down to the comments behind it. A summary built on structured data will point you at the right topics, ranked by impact. The analyst verifies what it says by reading the verbatims and adds the business context the summary doesn’t have.
What is the difference between summarising feedback and analysing it?
Summarising condenses. Analysing structures, counts and compares. A summary tells you returns complaints are up. Analysis tells you by how much, among whom, since when, and lets you open the mentions to see why. Good summaries are written on top of analysis, not instead of it.
How often should you summarise customer feedback?
Whenever there is a decision to inform, scoped to that decision. Because structured feedback is stored, a summary of this month can be compared with last month using the same topics and the same method, which is what makes the comparison meaningful.
SmartCX is the VoC platform that helps CX teams turn feedback data into CX intelligence the whole business can act on. Watch the product tour or talk to us about summaries you can trace back to the verbatim.