On this page
Concept testing means showing target users the concept for a new product or feature before you invest in building or launching it, hearing their first reaction, and then probing how it fits into their actual lives. The question it answers is not "Do users like it?" but "Who would use it, in what situation, and would they change what they do now for it?" After reading this guide, you'll be able to prepare concept materials, choose between a single-concept and multi-concept test, run interviews from one guide, and avoid two common problems: inflated purchase intent and participants telling you what you want to hear.
When to use it and how to set the research goal
- Before greenlighting a new product, to check whether the target audience really has this need.
- Before building a new feature, to choose between a few directions.
- Before launch, to settle on the lead selling point and messaging.
- Once a pricing or plan proposal is ready, to check whether users understand the value.
Write the research goal at the level of a decision, for example "Pick one of selling points A and B to lead with" or "Find out which type of user gets the most value from this feature." If the goal is "See what users think," it will be hard to make a call when you're done. Also write down in advance what result would make you drop the concept, so you don't only look for evidence that supports it.
How to prepare concept materials
The materials are there to help participants understand the concept, not to persuade them. A solid concept card usually includes:
- One sentence that says who it's for, what problem it solves and how.
- One specific usage scenario that describes what the person is doing, rather than a list of features.
- Key details: if you're testing purchase intent, include the price or price range and how it will be sold.
- One sketch or wireframe. It only needs to be understandable; no polished renders.
Keep the wording neutral. Remove adjectives like "revolutionary" and "best." When testing several concepts, keep their length, image quality and level of detail as similar as possible; otherwise you end up measuring differences in copy and design.
Single concept or multiple concepts
| Method | Good for | Watch out for |
|---|---|---|
| Single-concept test | You have one direction and want to know whether it's worth building and where the problems are | With nothing to compare against, reactions skew positive; calibrate them with past behavior |
| Multi-concept comparison | Choosing between two or three directions | Rotate the order; have participants evaluate each concept on its own, then compare after seeing them all |
In a multi-concept comparison, show no more than three concepts. By the fourth, participants struggle to tell them apart. Ask for a first reaction after each one instead of laying them all out at once and asking people to pick.
Interview guide
- [Opening] Thanks for joining. Today I'll show you an idea that's still being designed, and even our team disagrees about it, so telling me what doesn't work is just as useful as telling me what does.
- [Opening] Let's start with you: when was the last time you ran into (problem situation)? How did you handle it?
- [Opening] What don't you like about the way you do it now? Have you spent money or time on this?
- [Core topic: first reaction] (After showing the concept) Without any explanation from me, can you tell me in your own words what this is and who it's for?
- [Core topic: first reaction] What's the first thing that comes to mind when you see it?
- [Core topic: relevance] When would you use it? Give me a specific example from recently.
- [Core topic: relevance] Compared with how you do it now, what's better about it, and what's more of a hassle?
- [Core topic: concerns] Was there anything you didn't understand or that made you hesitate?
- [Core topic: purchase intent] If you could buy it tomorrow for (price), what would you do?
- [Core topic: comparing concepts] (After seeing all of them) If you could keep only one, which would you pick? Why not the others?
- [Probes] If you hear "It's nice" or "It's interesting": What specifically makes you feel that way?
- [Probes] If you hear "I'd use it": The last time you were in that situation, what did you actually do?
- [Probes] If they misunderstand: How do you picture this feature working? Don't correct them on the spot; note it down.
- [Probes] If you hear "I'd buy it": What would you do first? Who would you check with? What would you compare it to?
- [Closing] If you were in charge, what's one thing you would change?
- [Closing] Anything else you'd like to add? Thank you.
Ask for a first reaction before asking about purchase intent
Ask for the first reaction before any explanation. Having participants restate the concept in their own words shows whether your materials explain it clearly. If they restate it wrong, every later judgment rests on a misunderstanding, so note it separately.
Purchase intent is the easiest thing to inflate. Saying "I'd buy it" costs nothing, while actually buying means spending money and changing habits. Rob Fitzpatrick's advice in The Mom Test (2013) is to ask less about the imagined future and more about what has already happened. A few more reliable ways to ask:
- Ask about past behavior: Have you ever spent money on this problem? What workarounds have you used? When someone who has never paid or looked for a solution says they'd buy, discount it.
- Ask about next steps: If it launched today, what would you do first? Someone who can name a concrete action is more credible than someone who only says they'd "consider it."
- Ask for a small commitment: leaving contact details to hear about launch, booking a trial, joining a waitlist. These actions say more than stated intent.
- Ask with the price: ask about intent after participants see the price. Intent without a price is close to meaningless.
How to keep participants from just telling you what you want to hear
- Explain that the concept isn't final, and criticism is as useful as praise.
- The interviewer doesn't say "we designed this" and shows neither pleasure nor disappointment at answers.
- Don't ask leading questions like "Do you like it?" Ask "When would you use it?" instead.
- Have participants critique it on someone else's behalf: Who do you think wouldn't use this? Why?
- Treat compliments as background noise, and record only specific situations and past behavior.
Sample dialogue: from "I'd use it" to the real obstacle
This is a fictional example. The participant's first answer is "I'd use it," but probing past behavior shows the real obstacle is data entry. That tells you far more about what to do next than "I'd use it."
How to synthesize findings
- Start with comprehension: how many participants could restate the concept correctly, and where misunderstandings cluster. Example: 6 of 20 participants understood the feature as something else, which means the materials need revising.
- Then look at relevance: who needs it and in what situation, broken down by segment.
- Separate first reactions from judgments formed after probing, and flag anyone whose attitude changed after follow-up questions.
- Tier purchase intent by strength of evidence: people who have paid or used a workaround before, people who can name a concrete next step, and people who only say they would.
- For multi-concept comparisons, record both preference and reasons, with counterexamples for every concept.
- Tie findings to decisions: build or don't build, which option to pick, what to change.
Common mistakes
- Showing polished renders: participants will judge the visuals, not the concept. A sketch is enough.
- Explaining at length before asking for opinions: the more you explain, the less you can measure a genuine first reaction.
- Treating liking as need: liking something doesn't mean someone will use it, much less pay for it. Probe for past behavior.
- Not rotating concept order: whichever concepts come first and last get an advantage.
- Only recruiting loyal users: they are more forgiving of new things, which makes findings overly optimistic. Include non-users from your target audience.
- Treating stated purchase intent as a conversion rate: discount it based on past behavior and level of commitment.
Validate findings with behavior data
- Selection: use behavior data to find people who actually have the relevant behavior, such as users already relying on a workaround or using related features frequently, so you don't end up interviewing people outside your target audience.
- Before release: use a fake door test or a waitlist to measure real clicks and sign-ups, and compare them with the intent you heard in interviews.
- After release: set metrics based on the findings. For the chosen selling point, track product page conversion; for a new feature, track usage and retention.
- Compare words and behavior: re-interview people who said they would use it but don't show up in the data. You'll often find obstacles the concept test missed.
How to do it in heari
With heari, you give AI the business question and concept materials, and it drafts the study plan and interview guide. After the research lead reviews it, AI runs many interviews at once, asking for a first reaction and then following up on usage situations and past behavior based on each answer. You can select participants by behavior in Agentic Engine. In the insight report, each finding comes with quotes, counterexamples and participant counts, and after release you validate the findings with behavior data.
FAQ
How many people should you interview for concept testing?
For qualitative concept testing, start with about 12 interviews per target segment, based on Guest, Bunce & Johnson (2006), who found that about 12 interviews reach thematic saturation. If you need to compare how strongly different segments prefer different concepts, follow the interviews with a quantitative survey.
Should you test a single concept or compare multiple concepts?
Use a single-concept test when you have one direction and want to decide whether it's worth building, but calibrate the generally positive reactions against past behavior. Use a multi-concept comparison when you need to choose between two or three directions. Rotate the order, and have participants evaluate each concept on its own before comparing.
Can you trust participants who say they would buy?
Stated purchase intent tends to run high and can't be read as a conversion rate. More reliable signals are having spent money or found a workaround for the same problem in the past, being able to name a concrete next step, and agreeing to a small commitment such as leaving contact details or booking a trial.
How polished do concept materials need to be?
Just enough for participants to understand it: one sentence on who it's for and what problem it solves, plus a usage scenario and a sketch. Polished renders lead participants to judge the visuals rather than the concept. Keep the level of polish consistent across concepts.
