Someone on your team says five users is enough. They are quoting a 1993 model, and they are not quoting its parameter. In Laura Faulkner's 2003 study, random draws of five participants from a pool of 60 found between 55% and 99% of the known problems in the same interface, with the same tasks and the same moderator. The number you get depends on who walks in the door.
This guide is for researchers, product managers and CRO leads who have to defend a recruiting number before a study starts. By the end you will be able to pick one of four sizing methods, state the assumption it rests on, and say what your number buys you in problem coverage. It will not tell you to test more people. It will tell you when five is a defensible answer and when it is a guess.
Where the five-user number comes from
Nielsen and Landauer modelled problem discovery as N x (1 - (1 - p)^n): with N problems in the design, p the probability that any single participant hits a given problem, and n participants, the share of problems you find is 1 - (1 - p)^n. They measured an average p of 0.31 across their data set. At p = 0.31, five participants reach 84.4% coverage. That is where "five users find 85% of problems" comes from.
Two things get dropped in the retelling. First, the rounding: at p = 0.31 you need six participants to clear 85%, not five. Second, and more important, p is not a constant of nature. Virzi measured 0.32, Lewis measured 0.16 on a different system, and Spool and Schroeder found serious issues still surfacing after dozens of participants on open-ended e-commerce tasks. The honest version of the claim is conditional. Here is what five, ten and twenty participants find at four problem-occurrence rates:
| Problem occurrence rate (p) | Found with 5 | Found with 10 | Found with 20 |
|---|---|---|---|
| 0.31 (Nielsen and Landauer average) | 84% | 98% | 99.9% |
| 0.16 (Lewis) | 58% | 83% | 97% |
| 0.10 (uncommon issues) | 41% | 65% | 88% |
| 0.05 (rare, high-severity issues) | 23% | 40% | 64% |
Read the bottom row before your next planning meeting. The problems that kill a checkout are usually not the ones four out of five people trip over. Those got fixed years ago.
Method 1: fixed five, repeated often
Run five participants, fix what you find, run five more on the next build. Nielsen's argument was never that five is statistically sufficient. It was that the marginal user is worth less than the marginal study.
Use it when: you are iterating on a design that is still changing weekly, the interface is new enough that most problems are common (high p), and you can actually ship fixes between rounds.
It fails when: the study is a one-shot audit you will not repeat, the findings go to stakeholders who will treat "we found nothing else" as evidence of quality, or the product is mature and the remaining problems are rare by definition.
Method 2: size from a discovery goal
Invert the model. Pick the coverage you need (P) and the occurrence rate you expect (p), then solve n = ln(1 - P) / ln(1 - p).
| Coverage goal | p = 0.31 | p = 0.16 | p = 0.10 | p = 0.05 |
|---|---|---|---|---|
| 85% | 6 | 11 | 19 | 37 |
| 90% | 7 | 14 | 22 | 45 |
| 95% | 9 | 18 | 29 | 59 |
Use it when: the study has a compliance, legal or launch-gate purpose and you need to state coverage in writing, or when you are hunting a specific class of low-frequency failure such as an error-recovery path.
It fails when: you have no basis for p. Picking 0.31 because it is the published average is the same guess as picking five, wearing a formula as a costume.
Method 3: estimate p from your own first round, then resize
This is the method we default to, because it replaces the guess with your own data.
- Run an initial round of five. Log every distinct problem and mark which participants hit it.
- Calculate the observed p as the mean of (participants who hit the problem / participants run) across all distinct problems.
- Deflate it. Small initial samples systematically overestimate p, because problems only one person hit are the ones you have not met yet. Hertzum and Jacobsen documented this inflation. A practical correction is to drop the problems seen by exactly one participant from the numerator when re-estimating.
- Feed the deflated p into the table above at your coverage goal and recruit the difference.
Decision points: if deflated p lands above 0.25, stop at five to eight and spend the rest of the budget on a second study of a different flow. If it lands between 0.10 and 0.25, extend to 12 to 18. If it lands below 0.10, your tasks are probably too open-ended or your segment is too broad, and adding participants will not fix either. Re-scope before you re-recruit.
It fails when: you cannot recruit incrementally, or the study is moderated by different people across rounds, which changes what gets recorded as a distinct problem.
Method 4: size by behavioural segment
The model assumes one population. Most products in this region do not have one. A Turkish retailer selling domestically, cross-border into the EU and into the Gulf has at least three groups with different mental models, different payment expectations, and in the Gulf case a right-to-left interface. Nielsen's guidance here is 3 to 4 participants per group at two groups, and 3 per group at three or more. That is a floor, not a target, and it means a "five-user study" across three segments is really three underpowered studies.
Two rules worth applying before you write the screener:
- Segment on behaviour, not on demographics. First-time buyers and repeat buyers hit different problems. Ages 25-34 and 35-44 usually do not.
- Treat assistive-technology users as their own segment if you sell into the EU. The European Accessibility Act has applied to consumer-facing e-commerce services since 28 June 2025, and a sighted-only sample cannot tell you whether you comply. Screen-reader and keyboard-only participants find a class of problem that no amount of extra sighted participants will surface.
Which method fits your study
| If this is true | Use | Typical n |
|---|---|---|
| Weekly iteration, design still moving | Method 1 | 5 per round |
| One-shot audit with a written coverage claim | Method 2 | 11-29 |
| You can recruit in waves and want the cheapest defensible number | Method 3 | 5, then resize |
| Two or more distinct behavioural groups | Method 4 | 3-4 per group, minimum |
| You need a metric with a confidence interval, not a problem list | None of these | 20+ (see below) |
Worked example
This example is illustrative, not a client case. Say you are sizing a checkout study for a mid-market Turkish fashion retailer, domestic traffic only, redesign already live. Method 1 is out because the design has shipped, and Method 2 needs a p you do not have, so: Method 3 seeded by Method 4.
Two behavioural segments, guest checkout and logged-in repeat buyers, four participants each. Round one produces 19 distinct problems, 11 of them seen by exactly one participant. Raw p is 0.28; after dropping the singletons from the numerator it falls to about 0.17. At a 90% coverage goal and p = 0.17 the table says 14. You have 8, so you recruit 6 more, weighted toward the segment with the thinner findings.
Total: 14 participants, with a written reason for the number. [INTERNAL DATA NEEDED: observed p values from Switas's own e-commerce checkout studies, to replace the illustrative 0.17 with a benchmark range by vertical.]
Where this breaks down
The binomial model assumes problems are independent and that p is stable across participants. Neither is true. One confusing label causes three downstream failures, and a participant fluent in marketplace UI patterns hits a different set of problems than one who is not. Treat the numbers above as planning tools, not measurements. Woolrych, Cockton and Caulton have all argued that a single p misrepresents a population made of sub-groups.
This guide also does not cover:
- Quantitative benchmarking. If you want a task-success rate with a confidence interval, you are answering a different question. Plan at least 20 participants, and more for tight intervals.
- Card sorting and tree testing. Nielsen suggests at least 15 per user group. Problem-discovery math does not apply.
- Eyetracking. Around 39 participants for stable heatmaps.
- A/B tests. Sample size there is statistical power on a conversion metric, not problem discovery. Different formula, different logic.
One more limit worth naming: task design moves p further than n does. Five participants on a sharp, realistic task will out-find twelve participants on a vague one. If your sizing conversation is longer than your task-writing conversation, the ratio is wrong.
FAQ
Is the five-user rule wrong?
No, it is conditional. It holds at a problem-occurrence rate near 0.31 and a coverage goal near 85%. Outside those conditions it under-delivers, and nothing in the original work claimed otherwise.
How do I estimate p before I have run anything?
You mostly cannot, which is why Method 3 exists. If you must guess, use 0.30 for a new or heavily changed interface and 0.15 for a mature one, then correct after round one.
Does moderated versus unmoderated testing change the number?
Not the math, but it changes what counts as a problem. Unmoderated sessions tend to miss the cause behind a failure, which inflates the count of distinct problems and deflates your observed p.
What if I can only recruit five people?
Run five, and report coverage honestly rather than reporting the problem list as complete. Say which segments you did not cover. An under-sized study with stated limits is useful; an under-sized study presented as exhaustive is not.
Do I need more participants for accessibility testing?
You need different participants. Assistive-technology users are a separate behavioural segment with their own floor of 3 to 4, not an add-on to an existing sample.
Can AI-generated synthetic participants replace some of this sample?
They can help you rehearse tasks and screeners. They cannot produce a problem-occurrence rate, because p is a property of real behaviour in a real population. Do not put synthetic sessions into the denominator.
Does the same math apply to B2B or public-sector products?
The math does. The recruiting does not. Narrow B2B and public-sector segments often cap out at six or eight reachable participants, in which case state the cap as a study limitation rather than pretending the number was chosen.
Run this on your next study
Take the study you are about to brief, write down which of the four methods you are using and the p you are assuming, and put both in the research plan. If neither survives being written down, that is the finding. If you would rather have the sizing, screening and segment design handled for you, talk to Switas about your next research round.
Sources
- Nielsen, J. and Landauer, T. K. (1993). A mathematical model of the finding of usability problems. INTERCHI '93, 206-213.
- Faulkner, L. (2003). Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, & Computers, 35(3), 379-383.
- Spool, J. and Schroeder, W. (2001). Testing web sites: five users is nowhere near enough. CHI '01 Extended Abstracts.
- Sauro, J. A brief history of the magic number 5 in usability testing. MeasuringU.
- Sauro, J. Sample sizes for usability studies: one size does not fit all. MeasuringU.
- Nielsen, J. Why you only need to test with 5 users. Nielsen Norman Group.
- Nielsen, J. How many test users in a usability study? Nielsen Norman Group.
- Directive (EU) 2019/882 on the accessibility requirements for products and services (European Accessibility Act), EUR-Lex summary.
