Every call about AI feedback analysis has the same shape. Is it accurate? How is it different from ChatGPT? Is it a black box? What happens to my analysts? Where does the data go? Who carries the can when it gets something wrong? Most of the answers come back to one test. Can you open any number and see the words behind it? If you can, the rest mostly takes care of itself.
Key takeaways
- No AI gets text analysis completely right. Neither would a room full of people. The question is whether you can check it
- The difference between a feedback platform and ChatGPT is a stored, structured dataset. One counts. The other writes
- ‘How do you know?’ is the question a board or a regulator will ask. Your platform has to answer it for you
- Analysts keep their jobs and lose the tagging
- You are liable for what you do with the analysis, which is exactly why the analysis has to show its work
We have sat through a lot of these calls. There is a shortlist, a budget, and a room full of people with reasonable doubts. The doubts arrive in a predictable order, so here they are, in that order, with the answers we actually give.
The first question is always about accuracy
Is it accurate? Not completely. Nobody’s is.
Two people reading the same comment disagree about a fifth of the time. There is no right answer to mark against, so any percentage a vendor quotes is measured against somebody’s opinion. Anyone promising 100% is selling.
What you can rely on is consistency. One method, every comment, every source. Every mention stored with its topic and sentiment, so March and June were measured the same way and the trend between them is real. And you can check it yourself, by opening any number and reading what is underneath. The longer, more honest version is in how accurate is AI feedback analysis.
The second is always about ChatGPT
Or Copilot, or whichever assistant someone in the room used last week. Why can’t we just do it with that?
Because ChatGPT reads text and writes text. A feedback platform turns text into data. Paste 500 comments into a chat window and you get a plausible summary. Nothing stored, nothing counted the same way twice, nothing to click into. Ask again next month and you start from nothing.
A platform breaks every comment into mentions, files each one under a topic, scores its sentiment and keeps all of it. Now you can count, compare and drill down. Same sort of language models doing some of the reading. Completely different result, because there is a dataset underneath. The DIY route gets its own answer in where ChatGPT works and where it breaks, and the Microsoft version in why ‘it’s already in Excel’ isn’t the point.
Then someone asks whether it’s a black box
Usually the person who has to sign things off. Their real question is what happens when the chief executive, or the regulator, asks how you know.
The answer should be: you show them. In SmartCX, filing a mention under a topic is a matching step. The meaning of the phrase is compared with the set of topics, rather than a model being asked to write an answer. A second pass re-reads mentions against the full topic set to catch anything misfiled. Nothing is invented at that step.
Models still do the extraction, the sentiment and the summaries. Which is exactly why every number opens to the comments behind it, with the sentiment on each phrase. That is drilling down to the verbatim. It turns ‘the AI said so’ into ‘here are the 214 comments’, and it is the difference between a number you present and a number you defend.
The analysts want to know if they still have jobs
Nobody says it. Everybody is thinking it.
Yes. What goes is the tagging. Read a comment, pick a category, next comment, four thousand times. That is the part of the job they hated, and nobody who has done it will mourn it.
What stays is everything the tagging was in the way of. Checking what the platform found. Adding the context it doesn’t have, the courier change, the new store manager, the app release. Investigating the top driver until there is a process to change. Deciding which of three drivers gets the team this month. Following the fix through to whoever owns it.
That is the job. It just used to come with an admin tax. How to bring a nervous team with you is in the four fears about AI in CX teams.
Somebody has an example comment ready
There is always one. ‘Lovely food, slow service.’ What does the AI do with that?
Two things, because it scores sentiment on each phrase rather than on the whole comment. Positive mention for the kitchen. Negative mention for the floor. Score it as one comment and it comes out neutral, which is wrong twice.
A few comments will always be ambiguous, and no platform reads every one the way you would. Which brings us back to the drill-down. A person opens the topic, reads the comments, and catches the odd one before a decision rests on it.
Then they want to know who builds the topics
Not you. Topics are discovered in what your respondents actually wrote, not pulled from a library built for the industry or for somebody else’s business. So they sound like your customers rather than a consultant. And because ‘packed in like sardines’ and ‘really crowded’ mean the same thing, they land in the same topic even though they share no words.
When the business needs to track something specific, a new product line, a policy change, a competitor’s name, your team adds a category and it applies across every source from then on. Discovered topics tell you what people talk about. Your own categories tell you what the business is watching.
Other languages come up here too. Every response is translated into one language before analysis, so one set of topics and one method apply across the whole dataset. Wait times in Cardiff and wait times in Copenhagen count towards the same driver.
The operations people ask what it can eat
Open-text from anywhere. Surveys, reviews, support tickets, social posts and transcripts, brought in through CSV, API, webhooks or native integrations. Every response carries its context, who said it, on which channel, from where, so you can slice by anything you attach.
Then they ask whether it goes into Power BI, and the answer is yes. SmartCX connects to BI tools, so driver trends, topic volumes and sentiment sit next to churn, revenue and complaint counts on the reports leadership already reads. It also connects to case management, workflows and your CRM, so an individual problem reaches someone who can fix it while the person is still a customer. Some of that is native, some is through the API. SmartCX gives every team one trusted data foundation for every dashboard, decision and action, and the point of connecting it is that everyone works from the same numbers wherever they look.
The data protection officer has been waiting
Where does the data go?
Your data stays contained and separate from other clients. SmartCX offers UK, EU and US regional hosting, so it sits where you choose. Certifications include ISO 27001, Cyber Essentials Plus, FSQS and the NHS Data Security and Protection Toolkit, with GDPR compliance throughout. If your data protection officer wants more than a paragraph, they should, and that conversation happens before the demo, not after it.
Someone senior asks who is liable when it’s wrong
You are.
The software didn’t present the number to the board or send the letter to the tenant. Your organisation did. Which is the entire argument for traceability. If you are accountable for a decision, you need to show how the number behind it was reached, and a person needs to have looked before the decision was made.
So agree up front what gets a human sign-off. Anything that changes pricing, pulls a product, funds a programme or goes to a regulator. Keep the drill-down one click away for the moment someone asks how you know. Checking properly is a discipline, and how to validate AI insights sets it out.
And finally, how hard is it to set up
Less hard than the last one. Connect a source, usually your surveys, and run it on last year’s responses first. You already know what happened last year. So you can see straight away whether the drivers it surfaces are the ones you spotted, the ones you missed, or nonsense.
That is a better test than any demo dataset, because the demo dataset was chosen to look good and yours wasn’t. From there, your team gets product-trained support by phone, chat and email, and a trial run on your own feedback rather than ours.
That’s how SmartCX is built. Topics discovered from your own feedback, sentiment scored on every phrase, one method across every source, and any number one click from the comments behind it. See it on real feedback in our product tour.
SmartCX is the VoC platform that helps CX teams turn feedback data into CX intelligence the whole business can act on. Watch the product tour or talk to us with the questions we haven’t answered.