What Subject Lines Actually Earn Clicks? An Analysis of 12 Months of Open Data
Beehiiv and Substack show open rate per email. They don't tell you why. Anna classifies a year of subject lines by theme and surfaces the gap between what gets opened and what actually gets clicked
By Anna·~7 min read·Updated May 16, 2026
You sit down on a Sunday afternoon to write next week's newsletter, and the first thing you do — before the opening line, before the structure — is stare at the subject line field for twenty minutes.
You know openly that the subject line matters more than anything else in the email. You know just as clearly that you have no real method for choosing one. So you pick the kind of subject line you wrote last time that did well, or you copy the format from another newsletter you admire, or you write something contrarian because contrarian felt good to write.
Then you check the open rate three days later. It's 41%. Last week was 47%. The week before that was 38%. You shrug. The data is right there in your Beehiiv or Substack dashboard, telling you the number without telling you anything else.
The hidden question
Newsletter dashboards report open rate per email. They do not answer the actual operating question, which is:
Across a year of sending, which themes of subject line — story-led, contrarian, listicle, case study, curiosity gap, urgency — drive the most clicks, not just the most opens, and how does that depend on the size of my list?
That is a thematic question over a year of data. It cannot be answered by sorting last month's emails. It cannot be answered by A/B testing — A/B testing tunes copy within one email, not patterns across hundreds. And every newsletter platform refuses to do it for you because none of them have a category for "type of subject line."
The reason it matters is that the highest-opening subject lines and the highest-clicking subject lines are almost never the same. Opens are a measure of curiosity. Clicks are a measure of payoff. A subject line that promises more than the email delivers will out-open and under-click, and that gap is a signal about your reader's expectation — usually a useful one.
What Anna does
Anna runs the analysis in four steps. The end-to-end takes about fifteen minutes from the moment you export the data.
Step 1. Export from your ESP. Beehiiv, Substack, ConvertKit, Ghost — each will export a CSV of every email you've sent: subject line, send date, list size, opens, clicks, unsubscribes. Upload it, or use the appropriate integration if your platform has one. Anna sees one row per email.
Step 2. Classify each subject line by theme. Anna adds a theme column to the dataset using =AI("Classify this subject line into one of: story-led, contrarian, listicle, case study, curiosity gap, urgency, other", subject_line). The classification persists in the data as a column. Every future send gets classified automatically when the dataset refreshes. The taxonomy can be tightened — six themes is a starting point — and the column re-runs if you change the prompt.
Step 3. Compute the right metrics. Open rate alone is a trap because it conflates two different things: did the subject line earn attention, and did the email earn engagement once opened. Anna computes both: median open rate per theme, median click rate per theme, and the crucial derived metric — clicks per opener, which measures whether the email delivered on what the subject line promised.
Step 4. Segment by list size. A 1,000-subscriber newsletter and a 100,000-subscriber newsletter live in different worlds. Open rates compress as lists grow; clickthrough patterns shift; what works at one scale fails at another. Anna segments the analysis by list-size cohort so you're benchmarking against your own size band, not against survivorship-biased megalists.
What the report looks like
The output is one URL with three views. They map onto the three useful sub-questions: which themes earn attention, which themes earn payoff, and where the gap between the two tells you something.
The first view is the open-vs-click scatter. Each marker is a theme. The x-axis is median open rate, the y-axis is median click rate. The size of the marker is the number of sends in that theme.
The shape of the chart is what matters. Each quadrant tells a different story.
Top-right is the goal: high open, high click. In the example above, story-led subject lines sit there. They earn attention with a specific narrative hook and they deliver on it. The reader opens because they want to know what happens, and clicks because the email rewarded the curiosity with something to act on.
The bottom-right is the trap: high open, low click. Curiosity gap subject lines almost always live here. "You won't believe what happened with our last launch" out-opens almost everything but the click rate is below half of story-led. The reader felt baited, opened anyway, and bailed. Over time that trains them to ignore you.
The top-left is the underused quadrant: lower open, high click-per-opener. Contrarian subject lines tend to sit here. The opening pool is smaller — not everyone wants to read a contrarian take on a Monday morning — but the people who do open are committed, and the clickthrough among them is the highest in the set.
The second view is the full theme ranking with the click-per-opener metric foregrounded.
| Theme | Sent (12mo) | Median open | Median click | Click / opener | Net rank |
|---|---|---|---|---|---|
| Story-led | 18 | 48.7% | 7.2% | 14.8% | 1 |
| Case study | 22 | 51.3% | 5.4% | 10.5% | 2 |
| Contrarian | 9 | 33.8% | 6.1% | 18.0% | 3 |
| Curiosity gap | 24 | 44.6% | 3.6% | 8.1% | 4 |
| Listicle | 14 | 38.2% | 4.1% | 10.7% | 5 |
| Urgency | 7 | 42.1% | 3.8% | 9.0% | 6 |
Sort by click-per-opener and the ranking inverts the open-rate ranking. The case-study subject lines that drag everyone in produce middling engagement once they're in. The contrarian subject lines that filter out the lukewarm reader produce engaged readers. The listicle and urgency themes — the two that newsletter advice columns recommend most — are mid-pack on both metrics and bottom-pack on the net rank.
The third view is the list-size segmentation. Anna runs the same theme analysis cut by your subscriber count at the time of each send. Story-led wins at small scale because the writer's voice is the product. Case-study wins at large scale because the audience treats the newsletter as a research feed. The handover happens at a different size for every operator. Anna shows you yours.
What an operator does with this
The report changes the Sunday-afternoon subject-line problem from a creative gamble to a constrained choice.
The first action is to stop writing the themes that lose. Urgency rarely earns the click-per-opener it costs in reader fatigue. Pure listicle subject lines flatten the writer's voice into a format and the dashboard punishes them. You don't have to retire them entirely — sometimes the body of the email genuinely is a listicle — but they should be the exception, not the default.
The second action is to write more of the theme that wins for your size and stage. If you're a story-led writer at 3,000 subscribers, lean into it; the chart confirms what your gut already suspected. If you're at 30,000 and case studies are dragging your dashboard, that's the format the audience wants; structure more emails that way.
The third action is to read the gap between opens and clicks as a diagnostic, not a metric. When a curiosity-gap subject line earns a 47% open rate and a 3% click rate, the gap is the message: the email did not deliver. Either rewrite the email or rewrite the subject line. The dashboard will not surface this; the report will.
The deliverable is the report URL. You pin it in your newsletter notes. You open it on Sunday before you sit down to write. You stop arguing with your own gut about subject lines and start choosing them against a year of your own evidence. The first month of doing this is when the open-rate variance starts to compress around a higher median, because you stopped trying ten different themes a month and started running three that work.
Frequently asked questions
How many emails do I need before this analysis is meaningful?
The themes need at least five sends each to produce a median you can trust. With six themes, that's roughly thirty emails minimum — about six months of weekly sending. Below that, Anna will fall back to a directional read (which themes look stronger or weaker) rather than precise percentages, and will flag the cohort sizes that are too small to rank confidently.
What if my open rates are unreliable because of Apple Mail privacy?
Apple Mail Privacy Protection inflates opens for Apple users, which has been making open rates noisier since 2021. Anna can split the analysis by mail client when your ESP exposes that data; if it doesn't, she'll use click rate as the primary metric and treat open rate as supporting evidence. The click-per-opener metric is most affected by this and Anna will tell you when the numbers are likely overstated.
Can I add my own theme categories instead of the default six?
Yes — the classification lives as an =AI() formula Anna wrote into your dataset, and you can edit the prompt to change the theme list. If your newsletter operates in a specific niche where the useful taxonomy is different — "tweet-storm summary, link round-up, editorial, AMA recap" rather than the defaults — edit the prompt, re-run the column, and the report updates. The taxonomy is a parameter, not a fixed list.
How is this different from A/B testing my subject lines?
A/B testing answers "for this specific email, which of two variants performs better." Thematic analysis answers "across hundreds of emails, which patterns work for your audience." They're complementary: use thematic analysis to choose the format, A/B testing to refine within it. Most newsletter operators do only A/B testing and wonder why the gains plateau — they're optimising within a local maximum the thematic analysis would have moved them out of.
Does the analysis work for sponsored sends or paid placements differently?
Sponsored sends often have constraints on subject-line copy that make the theme classification noisier — the sponsor specifies the language, so the theme reflects the sponsor's voice more than yours. Anna lets you tag rows as sponsored or organic in the dataset and the report shows the two segments side by side. Most operators find their organic and sponsored theme winners are different, and the gap is worth knowing.