On this page
Qualitative research answers why and how something happens; quantitative research answers how many and how big the difference is. Use them together, and you learn where the problem is, what's causing it, and how many people it affects. By the end of this article, you'll be able to tell which method to start with for the question in front of you, plan a study using one of three common sequences, and have a decision checklist for choosing a research method.
Definitions and differences
Qualitative research gathers detailed material from a small number of people through interviews, observation, open-ended records, and similar methods, and draws out what situations exist and why they happen. Quantitative research gathers structured data from a large number of people through surveys, behavior data, experiments, and similar methods, and uses statistics to describe scale, compare differences, and test cause and effect.
| Criteria | Qualitative research | Quantitative research |
|---|---|---|
| Question it answers | Why, how it happens, what types exist | How many, what proportion, whether differences are significant |
| Common methods | User interviews, usability testing, diary studies, field observation | Surveys, behavior data analysis, A/B testing |
| Data type | Quotes, session recordings, observation notes | Numbers, options, events, and metrics |
| Sample size | Usually a dozen to a few dozen people | Usually several hundred or more, depending on the size of the difference you need to detect |
| Analysis | Categorize, compare, find patterns and counterexamples | Descriptive statistics, group comparisons, significance testing |
| Form of findings | What situations exist, and the reasons and conditions behind them | The scale of each situation, and the change a modification produces |
| Main risk | Treating individual comments as a widespread pattern | If the question is wrong, precise numbers don't help |
The difference isn't whether you use numbers. Interview findings also say things like "6 of 15 participants mentioned this" (example), but that number only describes the sample and can't be generalized as a proportion.
What each is good at
Qualitative research is good for
- When you don't yet know what the problem is, finding out what situations and types exist
- Explaining something you've already observed, such as why conversion is low at a particular step
- Understanding users' goals, contexts, and alternatives to inform design
- Preparing options for a survey, so they cover what users actually say
- Understanding small but important groups, such as major accounts or high-value churned users
Quantitative research is good for
- Estimating how widespread a need or problem is among users
- Comparing differences across user groups, channels, and versions
- Confirming whether a change actually moved a metric
- Prioritizing among several problems and deciding where to put resources first
- Tracking how metrics such as satisfaction and retention change over time
Three common ways to combine them
Qualitative first, then quantitative
Best when the problem is still fuzzy. Start with a dozen or so interviews to find the reasons and types users mention, then turn those types into survey options and survey a larger sample to measure how common each one is. For example, when researching why users don't renew, interviews might surface reasons such as price, using it less often, and switching to a competitor, and a survey then estimates the share of each. This way, the survey options come from users' own words, not the team's guesses.
Quantitative first, then qualitative
Best when you've already spotted an anomaly in the data but don't know why. First use behavior data or a survey to locate which step and which group the problem is in, then interview those people specifically. The advantage of this order is that you talk to exactly the right people, and the interview guide can be built around a specific step.
In parallel
Best when time is tight and you need both kinds of evidence. For example, after a new feature ships, look at usage data and A/B test results while interviewing users who did and didn't use it. When running both in parallel, agree in advance on how to compare the results: can the interviews explain the changes the data shows, and can the data show behavior that matches what keeps coming up in interviews? When the two don't agree, it usually means something is still unexplained, and it's worth digging one step further.
When choosing an order, first ask yourself whether you're missing an understanding of what's happening or an estimate of scale. If you're missing understanding, start qualitative; if you're missing scale, start quantitative.
Example: use behavior data to find the problem, then interviews to find out why
Here's an example. All numbers are made up to illustrate the process.
- Find: behavior data shows that new users drop off most at step three after sign-up, "link your account." For example, about 40% of users who complete step two don't complete this step.
- Locate: breaking it down by channel and device shows that drop-off is concentrated among mobile users from one ad channel.
- Select: from these users, find people who left at step three, plus a comparison group who completed it, and invite about 10 people from each group to interview.
- Interview: build the guide around what actually happened during their most recent sign-up, and probe what they saw at step three and what they expected to happen.
- Synthesize: group the reasons, label counts and counterexamples, and turn them into hypotheses you can test.
- Validate: change the page to address the main reason, and use an A/B test to compare the completion rate of the old and new versions.
An example exchange from an interview:
Example: 6 of 10 people in the churned group mentioned worrying about automatic charges, and 2 people in the comparison group had the same concern but still linked their account. This finding shows what the cause might be, but it doesn't show that 60% of all churned users left for this reason. The next step is to state clearly on the page that users will get a reminder before the trial ends and can cancel anytime, then run an A/B test to see whether the completion rate goes up. If you need to estimate how much this reason accounts for, add a single-choice question to the exit page.
How far samples and findings generalize
Both kinds of research have limits on how far findings generalize, and findings pushed past those limits are the most likely to mislead a decision.
- Qualitative findings generalize types and mechanisms, not proportions. They can show what situations exist and under what conditions they happen, but the sample isn't random, so proportions within the sample can't be applied directly to all users.
- Sample size in qualitative research depends on whether themes have reached saturation. Guest, Bunce & Johnson (2006) found that in a homogeneous group, few new themes emerge after about 12 interviews. When participants vary widely, each group needs enough people on its own.
- Nielsen's (2000) five-user finding applies only to finding usability problems, not to interviews or estimating proportions.
- Quantitative findings hold only for the population that was sampled. A survey sent only to active users can't be generalized to churned users, and a voluntary survey skews toward people with strong opinions.
- Behavior data covers all users, but it only records the behavior that event tracking captures and can't see what happens outside the product.
- A/B test results hold only for the test period, the people in the test, and the variants you compared. A different season, market, or audience needs to be validated again.
How to choose a research method
- What decision does this study need to support?
- Am I missing understanding (why, what situations exist) or scale (how many, how big the difference)?
- Can behavior data already pin the problem down to a specific step or group? If so, look at the data first.
- Do you need to understand users' motivations, expectations, or alternatives? If so, plan interviews.
- Do you need to watch users actually doing the task? If so, run usability testing.
- Do you need to estimate proportions or compare groups? If so, use a survey or behavior data, and confirm the sample covers the target population.
- Do you need to confirm the effect of a change? If you have enough traffic, use an A/B test.
- Where will the survey options come from? If there's nothing to base them on, run a few interviews first.
- Could participant counts in interview findings be misread as proportions? Does the report state who was in the sample?
- If the two kinds of results don't agree, how will we investigate next?
- If we only have time and budget for one method, which would most affect this decision?
Common mistakes
- Holding qualitative research to quantitative standards: dismissing findings because the interview sample is small. The value of qualitative research is in finding types and reasons, and its sample size is judged by saturation, not statistical significance.
- Reporting proportions from qualitative results: turning "6 of 10 people" into "60% of users." When you need proportions, estimate them separately with a survey or behavior data.
- Survey options based purely on team guesses: they miss users' real reasons, and the data looks complete but answers the wrong question. Collect users' own words through interviews first, then design the options.
- Looking at data without asking why: you learn that drop-off happens at step three and go straight to changing the button color. First find out what users ran into at that step, then decide what to change.
- Picking one side when results conflict: choosing the evidence you'd rather see. A conflict means something is still unexplained, so go back to the raw records and data definitions and dig further.
How to do it in heari
In heari, you can start by finding the drop-off point or anomaly in behavior data in ThinkingAI Agentic Engine and select participants by behavior. AI then runs many interviews at once by text or voice and asks follow-up questions based on the answers. Every insight in the report comes with quotes and counterexamples and is labeled with participant counts, and the research lead reviews each one. After the change ships, go back to the behavior data to check whether the findings hold.
FAQ
Is qualitative or quantitative research more reliable?
Neither is more reliable; they answer different questions. Qualitative research reliably shows what situations exist and why, and quantitative research reliably shows scale and differences. Reliability depends on whether a method is used for questions it can answer, and whether the sample covers the people you want to study.
If we interviewed 20 people, can we say what percentage of users have this problem?
No. Interview participants are usually selected by criteria, not randomly sampled, so proportions within the sample can't be generalized to all users. You can write a finding as "8 of 20 participants mentioned this" and state who was in the sample. When you need a proportion, estimate it with a survey or behavior data.
With a limited budget for only one kind of research, how should you choose?
It depends on what you're missing. If you can already see the problem in the data but don't know why, do interviews. If you already know the reasons but not which one matters most, run a survey or look at the data. To confirm the effect of a change, run an A/B test. If you can't say what you're missing, start by using existing behavior data to pin down the problem.
When behavior data and interview findings don't match, which should you trust?
Don't rush to pick one. Common causes include: the interview participants and the users in the data are different groups, what participants say differs from what they actually do, or the event tracking is defined differently from how people understand it. Checking the sample, raw records, and metric definitions one by one often leads to a new explanation, and that is a valuable finding in itself.
