Skip to content
hey annahey anna
Back to blog

We Ran 2,000 Customer Reviews Through AI. Here's What No One on the Team Had Noticed.

case-study
customer-feedback
sentiment-analysis

Every company has the same ritual.

Once a month, someone on the team opens the reviews dashboard, skims a few pages, screenshots one glowing review and one angry one, and drops them in Slack. Everyone reacts. Maybe someone says "we should look into that." Nobody does. Next month, repeat.

The signal is sitting right there — in 2,000 reviews your customers wrote for free. But nobody has time to read all of them, let alone quantify what they say. So you get anecdotes instead of evidence, and the product roadmap stays driven by gut feel.

Here's what happens when you stop skimming and start analyzing.

The dataset

Take a realistic DTC skincare brand — five products, roughly 2,000 customer reviews exported from Shopify as a CSV. Four columns: product name, star rating, review text, and date.

This is the kind of file that sits in someone's Downloads folder for weeks. It's not messy enough to need cleaning, but it's too big to read manually. The classic "we have the data, we just don't have the insight" situation.

Upload it to heyanna and let Anna take it from there — from raw CSV to shareable report.

Step 1: Upload and orient

Drag the CSV into heyanna. Anna immediately reads the shape of the data — the columns, the types, the row count. No setup. No column mapping. No data cleaning step.

Here's what the raw data looks like — a standard Shopify review export:

ProductRatingReviewDate
Hydrating SerumThis serum changed my skin completely. Already ordered my second bottle.2025-11-14
Daily MoisturizerIt's okay. Does the job but nothing special. Feels a bit greasy.2025-12-02
Vitamin C BrightenerLove the results — my dark spots are fading after 3 weeks!2025-10-28
Daily MoisturizerFine product, absorbs slowly though. Probably won't repurchase.2026-01-15
Gentle CleanserPerfect for sensitive skin. The scent is lovely and it doesn't dry me out.2025-11-30
Night Repair CreamThe pump broke after a month. Product itself is decent.2026-01-08
Raw review data — four columns, 2,000 rows of free text. Good luck reading all of them.

You're looking at your reviews in a spreadsheet within seconds. Which is exactly where they were before, just in a different tab. The magic starts in the next step.

Step 2: Enrich with AI() formulas

This is the part that changes everything. AI formulas let you run a language model across every single row of your data — tagging, classifying, extracting — and store the results as new columns. Like a spreadsheet formula, but instead of math, it understands language.

Anna added four AI() formula columns:

Sentiment — classify each review as Positive, Neutral, or Negative:

=AI("Classify the sentiment as Positive, Neutral, or Negative", C2)

Theme — what is each review actually about:

=AI("What is the main topic: Product Quality, Packaging, Scent, Texture, Results, Price, or Shipping?", C2)

Repurchase signal — does the reviewer plan to buy again:

=AI("Does this review express intent to buy again? Return Yes, No, or Unclear", C2)

Key complaint — distill each negative review to its core issue:

=AI("Summarise the main complaint in under 10 words", C2)

Four formulas. Anna applied them to all 2,000 rows. A few minutes of processing, and now every single review has structured metadata that didn't exist before:

ProductRatingReviewSentimentThemeRepurchaseComplaint
Hydrating SerumThis serum changed my skin completely. Already ordered my second bottle.PositiveResultsYes
Daily MoisturizerIt's okay. Does the job but nothing special. Feels a bit greasy.NegativeTextureNoGreasy, nothing special
Vitamin C BrightenerLove the results — my dark spots are fading after 3 weeks!PositiveResultsYes
Daily MoisturizerFine product, absorbs slowly though. Probably won't repurchase.NegativeTextureNoAbsorbs slowly
Gentle CleanserPerfect for sensitive skin. The scent is lovely and it doesn't dry me out.PositiveScentYes
Night Repair CreamThe pump broke after a month. Product itself is decent.NegativePackagingUnclearPump broke after one month
The same reviews, enriched. Four new columns — sentiment, theme, repurchase intent, and complaint summary — all extracted by Anna.

On the left: raw text you'd never read in full. On the right: four clean columns that Anna extracted from language your customers already wrote — sentiment, theme, buying intent, and complaint summary turned into quantifiable data.

Start with sentiment and theme — those two columns unlock 80% of the insight. Repurchase intent and complaint extraction are powerful follow-ups once you see the patterns.

Step 3: Ask Anna to build the report

Now that every review is tagged, Anna has structured data where there used to be free text. Time to make sense of it.

Tell Anna: Build me a report on customer sentiment across our product line. I want to know which products are loved, which are struggling, and what's driving the negative reviews.

One prompt. Here's what Anna built:

Metrics row

The headline numbers at the top — instant orientation before you read anything else:

Positive sentiment
68%
across all products
Total reviews
2,047
Lowest repurchase
31%
Daily Moisturizer

Sentiment by product

The Hydrating Serum is overwhelmingly loved. The Daily Moisturizer is polarizing — nearly as many negative reviews as positive ones. This chart alone tells a product story that star ratings flatten out.

Hydrating SerumVitamin C BrightenerGentle CleanserNight Repair CreamDaily Moisturizer0100200300400
NegativeNeutralPositiveNumber of reviews
Sentiment distribution across the product line. The Daily Moisturizer has nearly as many negative reviews as positive ones.

Theme breakdown

Anna broke down the top themes per product and split each by sentiment. The pattern is immediate: when customers love a product, they talk about results — what happened to their skin. When they're unhappy, they talk about texture — how the product feels. "Daily Moisturizer — Texture" is a wall of coral. "Hydrating Serum — Results" is almost entirely teal. Different products fail for different sensory reasons.

050100150200250Daily Moisturizer — TextureDaily Moisturizer — ResultsNight Repair Cream — PackagingNight Repair Cream — ScentGentle Cleanser — ScentGentle Cleanser — ResultsVitamin C — ResultsVitamin C — PriceHydrating Serum — ResultsHydrating Serum — Packaging
NegativeNeutralPositiveNumber of reviews
Top themes by product, split by sentiment. 'Daily Moisturizer — Texture' is overwhelmingly negative; 'Hydrating Serum — Results' is overwhelmingly positive.

The hidden insight

This is the finding that justified the entire exercise.

The Daily Moisturizer has a 4.1-star average. By any normal dashboard metric, that's a solid product. It's above 4 stars. Ship it.

But the sentiment analysis tells a different story. When you read the actual words — not the star count — the Moisturizer has the lowest sentiment score in the lineup. Customers are giving it 4 stars and then writing things like "it's fine, nothing special" and "probably won't repurchase."

Star ratings and sentiment diverge. Customers are being polite in their ratings but critical in their words. They're not angry enough to leave 2 stars, but they're not coming back. "Politely disappointed" is the pattern Anna surfaces — and it's invisible in any dashboard that only tracks the number.

Repurchase intent by product

Anna extracted repurchase intent from review language alone. The Hydrating Serum hits 73% — customers write things like "already ordered my second bottle" and "this is a staple now." The Daily Moisturizer? 31%.

73%61%58%44%31%Hydrating SerumVitamin C BrightenerGentle CleanserNight Repair CreamDaily Moisturizer01020304050607080
Repurchase intent (%)
Repurchase intent extracted from review language. The Daily Moisturizer (red) is the clear outlier at 31%.

Star rating says 4.1. Review language says churn risk.

Top complaints table

The ten most common distilled complaints, ranked by frequency. "Greasy texture" and "absorbs slowly" cluster at the top for the Moisturizer. "Pump broke" flags a packaging issue across two products. These aren't cherry-picked quotes from one angry reviewer — they're patterns across hundreds of responses.

ComplaintProductFrequency
Greasy textureDaily Moisturizer87
Absorbs too slowlyDaily Moisturizer64
Pump mechanism brokeNight Repair Cream52
No visible resultsDaily Moisturizer41
Strong fragranceNight Repair Cream38
Pump broke within weeksVitamin C Brightener34
Caused breakoutsDaily Moisturizer29
Too expensive for sizeNight Repair Cream27
Packaging leaked in transitHydrating Serum22
Pilled under makeupDaily Moisturizer19
Top 10 complaints distilled from negative reviews. Five of the top ten belong to the Daily Moisturizer.

Anna's report doesn't just show you charts — it narrates the findings. Each section explains what the data means, not just what it shows. The "politely disappointed" insight came from Anna connecting the divergence between star ratings and sentiment scores.

Step 4: Share with the team

One click. The report gets a shareable link. The product lead sends it to the founder, the merchandising team, and the customer experience manager. No slide deck. No copy-pasting charts into Google Docs. No 45-minute meeting to walk through what you found.

Everyone opens the same link. Everyone sees the same structured findings — metrics, charts, insights, evidence. The conversation shifts from "I read some reviews and here's what I think" to "here's what 2,000 reviews actually say, quantified."

From CSV upload to shared report: under 15 minutes.

The punchline

The star rating said 4.1. The reviews said "I probably won't buy this again."

Without AI reading every single review and quantifying the patterns, that moisturizer stays in the lineup for another six months while customers quietly churn. The review ritual continues — someone skims, someone screenshots, everyone nods. The signal stays buried.

The report caught it in 10 minutes. Not because AI is smarter than your team — but because your team doesn't have time to read 2,000 reviews and your dashboard doesn't know the difference between a 4-star review that says "love it" and one that says "it's fine, I guess."

Your customers are already telling you what they think. You just need something that can listen at scale.

You have the reviews. You just need someone to read them.

Try it free

No credit card required