On this page
The core of ad testing and brand message testing is finding out what audiences remembered, how they understood it and what they misunderstood, not whether they liked it: let participants recall freely first, then give prompts, and finally probe comprehension and misunderstandings. After reading this guide, you'll be able to decide between a pre-test and a post-test, copy the interview guide as is, write up findings as specific revision recommendations, and validate them with campaign and behavior data.
When to use it
- Before finalizing a new ad, slogan or packaging, you want to know whether the key message gets across
- After launch, the ad's click-through rate is decent but conversions or sales are disappointing, and you want to know where the problem is
- After a brand refresh or name change, you want to know how users understand the new messaging
- You can't decide between several creative versions and need evidence, not personal preferences in the meeting room
Pre-test or post-test
| Criteria | Pre-test | Post-test |
|---|---|---|
| Timing | Before the creative is finalized, using rough cuts, storyboards or static mockups | After the campaign has run for a while, using the live creative |
| Question it answers | Whether the message is understood, and whether there are obvious misunderstandings | What audiences actually remembered, and why campaign data doesn't match expectations |
| Who to talk to | People in the target audience who haven't seen the ad | People who saw or may have seen the ad, plus people who clicked but didn't convert |
| Limitations | Participants watch on purpose, so their attention is more focused than in real life | Memory is affected by time and by other ads |
The two don't conflict. If the budget allows, use a pre-test to rule out obvious misunderstandings and a post-test to explain campaign data. If you can only do one: do a pre-test if the creative can still change; do a post-test if the ad is already running and you need to decide whether to keep it or swap the creative.
How to set the research goal
Aim the goal at specific messages, not at "evaluating ad performance." First write down the one or two things you want the audience to remember from the ad, called key messages; the research tests whether they come across.
- Example goal: test whether audiences can say "book online, no waiting in line" after seeing the ad, and whether they mistake it for something that costs extra
- Rank the message tiers: tier one is what must be remembered, tier two is good to remember, and tier three is background such as brand tone
- Spell out what the findings are for: picking a version, rewriting a line of copy, or adjusting the target audience
An ad's first tier should carry no more than two messages. If the team lists five messages that must be remembered, that's a problem to solve first.
Who to interview
- Focus on the target audience. Select people the ad is meant to reach, such as those who bought this kind of product in the past three months
- Split by behavior: users who clicked but didn't convert, users who saw it but didn't click, and users who already bought often understand the ad differently
- In pre-tests, avoid people who know internal information, such as employees, partners and heavy brand fans
- When comparing versions, rotate which version each participant sees first, so the first version doesn't get an advantage
Unaided recall first, then aided recall
The order can't be reversed. Unaided recall gives no cues and lets participants say what they remember on their own; aided recall shows the brand name, an image or a keyword and asks whether it rings a bell. If you prompt first, participants will follow the prompt and you can no longer measure unaided recall.
In a pre-test, have participants watch the ad once, talk about something else for a few minutes, then ask what they remember. In a post-test, start with the product category, ask which related ads they've seen recently, and then gradually narrow down to the one you're testing.
- [Opening] Thanks for joining. Today I'd like to hear what you think about some content. There are no right or wrong answers; just tell me how you understand it.
- [Opening] When you buy this kind of product, when and where do you usually buy it?
- [Core topic: unaided recall] Have you seen any ads for this kind of product recently? Which ones do you remember?
- [Core topic: unaided recall] (Pre-test: after watching the ad and talking about something else) What do you remember about the content you just saw?
- [Core topic: aided recall] (Show the brand or an image) Do you remember seeing this? Do you remember what it said?
- [Core topic: comprehension] In your own words, what is it trying to tell you?
- [Core topic: comprehension] Who do you think it's talking to? Is it you?
- [Core topic: misunderstanding] Was there anything you didn't understand or felt wasn't clear?
- [Core topic: credibility] Do you believe what it says here? Why or why not?
- [Core topic: action] After seeing it, what would you do next?
- [Probes] You said it feels "pretty premium." Which image or line gave you that impression?
- [Probes] How does this compare with the product you use now?
- [Probes] When you say "free," which part do you mean is free?
- [Closing] If you were telling a friend about it, what one line would you use?
- [Closing] Anything else you'd like to add? Thank you.
Ask what they remembered, how they understood it and what they misunderstood
These three questions point to three different problems: if something wasn't remembered, the message isn't prominent enough; if it was understood differently, the wording is the problem; if it was misunderstood, a line or image suggested a meaning you didn't intend. Below is a sample interview about an ad for a mobile phone plan. The plan and prices are fictional.
The participant remembered the visuals and understood "lots of data," but didn't recall the price, a key message, unprompted. Once prompted, another misunderstanding surfaced: they took the long-term price for a first-month discount. Both point to specific fixes: show the price earlier and more prominently, and state that it's the ongoing price.
Record by message tier
After each interview, record results message by message so you can tally them directly when you synthesize. The table below is the record for the sample interview above.
| Message | Level | Unaided recall | After prompt | Misunderstood |
|---|---|---|---|---|
| Lots of data | Tier one | Recalled | No prompt needed | None |
| Monthly price | Tier one | Not mentioned | Remembered | Thought it was a first-month discount |
| No speed caps | Tier two | Not mentioned | Still didn't remember | None |
How to keep interviews from turning into aesthetic ratings
- Don't ask "Do you like this ad?" or "How would you rate it?" Ask what it said and what it means to them
- When participants comment on visuals or music, ask what it made them think of, to steer aesthetic judgments back to how they understood the message
- When comparing versions, look at which version's key messages more people restate accurately, not which one is more popular
- The interviewer doesn't share their opinion of the creative or say which version is the final one
- You can record whether they like it, but only as a reference: an ad people like won't necessarily make them remember the brand or message
How to synthesize findings
- Tally each message: for every key message, count how many people recalled it unprompted, how many remembered it after a prompt and how many misunderstood it. Example: 6 of 20 participants mentioned the price in unaided recall.
- Categorize misunderstandings: what was misunderstood, and which line or image caused it.
- Break down by segment: do different groups remember and misunderstand different things?
- Include quotes and counterexamples with every finding.
- Write them up as revision recommendations: what to keep, what to cut, which message to move up, down to specific copy or visuals.
How to validate with behavior data
- Before launch: use past campaign and conversion data to find the pattern you need to explain, such as a creative with a high click-through rate but low conversion
- After changes: map each finding to an observable metric, for example whether moving the price up increases plan page conversion and reduces price-related inquiries
- Use an A/B test to compare old and new creative, rather than only looking at before-and-after changes, to avoid interference from seasonality and promotions
- After the ad has run for a while, do another round of unaided recall interviews to check whether the key messages are being remembered
Common mistakes
Common mistakes
- Asking whether they like it or how they'd rate it: you only get personal taste. Ask what they remembered and how they understood it.
- Prompting before asking about recall: recall results will follow the prompt. Do unaided recall first, then prompt.
- Having the creative team moderate the interviews: they tend to explain and defend without realizing it. Have the research lead or a neutral moderator run them.
- Testing only one version with no reference point: you can't tell whether the results are good or bad. Add the previous ad as a reference.
- Treating pre-test comprehension rates as campaign performance: participants pay closer attention in a pre-test, so results skew optimistic. Read them alongside campaign data.
- Focusing only on messages that were remembered: messages that went unnoticed or were misunderstood are often the ones that most need fixing.
How to do it in heari
In heari, you can select participants with behavior data from ThinkingAI Agentic Engine, for example users who clicked an ad but didn't buy. AI drafts an interview guide based on the key messages you want to test, and the research lead reviews and launches it. AI first asks participants to restate the ad, then follows up on understanding and misunderstandings based on their answers, running many interviews at once. The report shows participant counts, each finding comes with quotes and counterexamples, and after the creative is adjusted you validate with behavior data.
FAQ
How many people should you interview for ad testing?
As a reference, Guest, Bunce & Johnson (2006) found that after about 12 interviews with one type of person, few new themes emerge. Obvious misunderstandings often show up in the first few interviews. If you're comparing different groups or several versions, each group needs enough interviews.
What's the difference between unaided and aided recall?
Unaided recall gives no cues and looks at what participants bring up on their own; aided recall shows the brand, an image or a keyword and checks whether participants recognize it. The first shows how deep an impression the message left; the second shows whether the message was seen at all. Always do unaided recall first.
Can interviews replace A/B testing your ads?
No. Interviews explain why audiences understand an ad the way they do; A/B tests measure which version drives more conversions. A good approach is to use interviews to find problems and fixes, then use an A/B test to confirm the effect of the changes.
Why does an ad still underperform when every participant says they like it?
Liking an ad is different from remembering or understanding it. Participants may like the visuals and music but be unable to say what the ad was trying to communicate, or get it wrong. To judge an ad, look at whether the key messages can be restated accurately, not at how much people like it.
