November 19th, 2026

Usability testing: why five participants are enough (and what they miss)

Five participants find 91% of the most common problems — and only 38% of all of them. How to set up the test, what to do with the recordings, and when five is not enough.

Rodrigo Carvalho Rodrigo Carvalho

Usability testing: why five participants are enough (and what they miss)

You ran five sessions, wrote down twelve problems, and someone asked: “what if we had tested with fifty?”. The short answer is that fifty people would probably show the same twelve problems — plus a few rare ones you may never fix.

The trouble is that the five-user rule became a slogan and lost the condition that propped it up. It does not say five people reveal 85% of your product’s problems. It says something far narrower — and understanding that difference changes how you plan the test, how much time you reserve for analysis, and what you promise in the readout meeting.

The math that became a rule

The formula comes from Jakob Nielsen and Tom Landauer, in 1993: with n participants you find N(1-(1-L)^n) problems, where L is the probability that a single participant runs into a problem. With the average L of 31% measured across a range of projects, five participants reach 85%.

Look at what fell out of the slogan: those 85% are of the problems that affect roughly one third of users. Not 85% of all problems in the interface. Nielsen stated this explicitly in the original article — the recommendation was three rounds of five, fixing in between, not a single round.

The underlying logic still holds: common problems show up fast, rare problems need a lot of people. That is why a test with five is good for deciding what to fix this week and bad for claiming the product “has no more problems”.

What happens when you check the math against real data

The MeasuringU team did exactly that check against seven usability study datasets: 231 participants and 406 catalogued problems and insights. They then split the problems by frequency and measured how much the first five participants caught in each band.

Problem frequencyProblems in the 7 datasetsFound by the first five
Affects 31% to 100% of users3591% (32)
Affects 20% to 30% of users2763% (17)
Affects 5% to 19% of users16440% (65)
Affects up to 5% of users18022% (39)
All problems40638% (153)

The first five participants found 91% of the most common problems — and 38% of the total. In other words: the rule works for what it promises, and what it promises is far less than most teams advertise. The full analysis breaks down all seven datasets with the average frequency of each study.

This is not an argument for testing with 50 people. It is an argument against treating one round of five as a final audit. Five people tell you “the signup flow stalled for three out of five” — which is already reason enough to fix it tomorrow morning.

So how many people do I need?

The right question is “for which goal”. Three cases show up in practice:

  • Finding problems and understanding why: 5 participants per round, one round per prototype version. Teams that test often book 3 to 5 sessions per cycle. The NN/g recommends five for a qualitative study with a single user group.
  • Measuring, comparing versions or proving improvement: here you need numbers, and numbers from five people are noise. Quantitative studies usually need dozens of participants, and precision grows slowly: cutting the margin of error in half takes roughly four times the sample.
  • Covering very different profiles (newcomer, power user, administrator, long-time customer): each group is its own study. Five people in total, mixing profiles, hides exactly the differences you wanted to see.

If the test is with five, don’t run three tasks per participant — run short, specific tasks. A 60-minute session with four long tasks produces fatigue and silence, not insight.

The bottleneck is not the session, it’s the analysis

This is the part that blows up the timeline. An NN/g survey of 50 teams testing commercial websites for the first time measured 39 hours for the complete test: planning, writing tasks, recruiting, running five sessions, analyzing and writing the report. With experience, the same cycle fits into two work days — but the session itself was never the biggest slice.

Unmoderated testing does not solve it. It saves facilitation time, not analysis time: someone still has to watch (or read) five recordings and turn that into a decision. NN/g is blunt about it: whatever time you save running sessions tends to reappear in the analysis phase.

That is why the recording matters more than the recruiting tool. If you have to rewatch 5 hours of video to remember where participant 3 gave up, the test cost you double.

Think-aloud changes behavior — and almost nobody controls for it

Asking people to think out loud is the standard in this field, and the standard has a known cost. When the moderator relaxes the protocol and starts asking “what did you expect here?”, behavior shifts: one comparative study showed longer task times, more page-to-page navigation, more scrolling and higher mental workload during relaxed verbalization.

Hertzum’s review of the topic sums up the impasse: classic think-aloud may not change behavior, but it adds little beyond what you already see on screen; relaxed think-aloud adds information but changes performance. He suggests a practical way out — separate doing from commenting, using the recording for a retrospective interview. It is also worth knowing the size of the effect: in one of the studies, only 38% to 44% of verbalizations during relaxed think-aloud were of medium or high relevance for identifying problems. The rest is chatter.

Two practical consequences:

  1. If the goal is comparing task time or completion rate, let the person work in silence and comment afterwards, recording in hand.
  2. If the goal is understanding the mental model, think-aloud helps — but a trained moderator must know that every question contaminates the data that follows.

How to turn a recording into a finding

The flow below is what survives when time is short. It works with any recording tool, and step 2 is where most of the work disappears.

  1. Record screen and session audio. On a Mac, QuickTime saves .mov; on Windows, the Xbox Game Bar screen recorder saves .mp4; OBS saves .webm or .mkv. One file per participant, named with the profile and the date.
  2. Transcribe before watching. Reading is faster than listening, and you can search by word. A 60-minute session becomes text in a few minutes; transcribing long audio has duration and plan limits worth checking before you upload five files at once.
  3. Mark three things per participant, not thirty. Task outcome (completed, completed with effort, gave up), the friction quote (“I thought this button would save it”) and severity. Thirty findings in a report is the same as none: nobody fixes thirty.
  4. Group by step in the flow, not by participant. The finding is “3 of 5 never found the coupon field”, not “participant 2 got confused”.
  5. Record the decision. Each finding becomes one line with an owner and a next action — the same mechanism as any checklist that comes out of a meeting. A finding without an owner comes back next round.

In Sintesy, that path is upload the video, transcribe, summarize and check off. Accepted video formats include .mp4, .mov, .webm and .avi, so the QuickTime, Zoom or OBS file goes straight in. Once transcribed, you can ask the chat where each participant got stuck and compare the answers — without replaying five sessions.

An honest note on plans: on the free plan the uploaded file is capped at 20 MB, which usually means exporting audio only, and long transcripts come back truncated. To analyze five full sessions, the Pro plan accepts files up to 2 GB and does not truncate the output. If all you want is the map of the five conversations, free is enough; if you want the complete document to quote in the report, it is not.

If you need a number, use SUS — and not with five people

The System Usability Scale is the 10-question survey that returns a score from 0 to 100. The historical average across evaluated systems is 68; above 80 you are in what the literature calls good usability. It is a cheap way to get a comparable number across versions.

Just don’t ask SUS for what it cannot deliver. With five respondents the score is useful for an internal conversation, not for claiming a statistical improvement. If the decision depends on the difference between 68 and 74, the study stopped being qualitative and the sample size changed by an order of magnitude.

Mistakes that show up in almost every test

  • A task that gives away the answer: “click Complete purchase” is not a task, it’s an instruction. “Buy one item and have it delivered” is a task.
  • Instructions that only work live: in an unmoderated test there is no moderator to clear up the misunderstanding. Rewrite until the instruction stands on its own and pilot it first.
  • A moderator who explains: every explanation you give is a hypothesis you just killed.
  • No record of what has already been fixed: without a baseline, the second test proves no progress at all.
  • Recruiting from your own team: a colleague tests the product knowing too much. Five people from inside the company are not five users.

What five people actually prove

They prove friction exists in the most-used paths, and they show you where. They don’t prove the product is good, that rare problems are absent, or that the new version beats the old one.

In practice, that is enough to work with. The question that closes the test is not “how many problems remain?” but “what do we fix before the next round?”. Five people answer that question comfortably — as long as someone reads all five recordings and writes the answer down.