How many raters does 360° feedback need – and why two is only the floor

The short answer: two is the floor, three or more is the target

The question has two answers, and both are right. The first is about anonymity: how many people must have responded before nobody can work back from a group average to an individual answer? Here two is the hard floor – with a single response, the group score simply is that person's answer. The second is about quality: how many perspectives does an average need before it says something about the leader rather than about one person's mood that week? The honest answer: more than most programmes plan for.

DefinitionAnonymity threshold

The minimum number of completed responses at which a rater group (the peers, say) is reported separately. Below it, the group's answers are not discarded but folded into a pooled "all others" category. The threshold protects answers, not names: the leader usually knows who was nominated and who has submitted – not what those people said.

Why is the floor not enough? With two responses, each answer is half the average: if one rater scores a behaviour 2 and the other 6, the report shows a 4 – a value neither of them gave. From three people onwards that smooths out; from five or six it becomes noticeably more stable – statistically reliable only at the numbers the research cites below. How anonymity is produced technically, and where its limits are, is covered in Is 360° feedback anonymous?

What research says about the number of raters

Reliability research on 360° ratings agrees on one thing: you need more raters than are typically used. Greguras and Robie (1998) applied generalizability theory to 360° ratings of 153 managers and found little agreement within a source – among a leader's peers, for instance; much of the error variance was down to the individual rater and the specific rater–leader pairing. Acceptable reliability, they concluded, requires more raters than are typically used. Under conditions common in practice, peers were the most reliable source, then direct reports, then supervisors.

Nowack and Mashihi (2012) summarise that work in their evidence review: the optimum for most 360° projects is at least four supervisors, eight peers and nine direct reports to reach a reliability of .70 or higher – while acknowledging that this is impractical for leaders with few direct reports. When two or fewer respondents provide data for a group, they add, the sample may be inadequate for reliable measurement, so inviting more raters rather than fewer is advisable.

Hensel and colleagues (2010) arrived at similar magnitudes: roughly ten raters were needed to reach a reliability of .70 when rating the capacity to develop personal qualities, about six when rating the motivation to develop them. Two or three peer raters – common in HR practice – yielded low reliability and low supervisor–peer agreement. The authors conclude that 360° feedback works better as a development tool than as an administrative appraisal system.

How many raters we recommend per group

Floor, stability and reality combine into a rule of thumb we apply in programmes. It distinguishes between the number that must actually have arrived in the report and the number you should nominate – because not every nominated person responds.

How to fill those slots with the right people – not just the sympathetic ones – is covered in How to select raters for a 360°. The self-assessment does not count as a group: the self–other comparison, which research studies for its presumed links to self-awareness and leader effectiveness (Fleenor et al. 2010), is only as robust as the other-ratings behind it.

Why supervisors are a special case

Most leaders have exactly one supervisor. A group with one response cannot be anonymous: its average is the answer. Three clean ways exist. First: the supervisor's scores are folded into "all others" and never shown separately – the default unless agreed otherwise. Second: the supervisor knows in advance that their answers will be identifiable as a column of their own and explicitly agrees. In development programmes that is often exactly what is wanted, because leader and supervisor then start the conversation from the same numbers. Third: a second person with a genuine basis for observation – a functional line manager, a partner on the engagement, the managing director – lifts the group over the threshold.

What does not work: nominating a second person merely to make the number. Someone who does not regularly experience the leader dilutes the picture rather than sharpening it – and "cannot assess" becomes the most frequent answer.

Small teams: when the numbers fall short

What if a leader has only two direct reports – or one of them declines? Four options, in order of our preference:

1. Merge the group rather than drop it. The answers are folded into "all others"; the leader loses the separate view of the group, not the feedback.
2. Add nominations while the survey is still open. Former direct reports, people the leader leads on projects, the wider team – provided they have genuinely experienced the leader.
3. Allow voluntary feedback. Some programmes let people in the same cycle give feedback without being nominated. That raises the number but needs clear rules on who qualifies.
4. Drop the group deliberately and say so in the debrief. With a single direct report, a 360° is really a 270°. No reason not to run it – but a reason to call it what it is.

In consulting and professional-services firms the problem runs the other way: project leaders have many temporary reports and rarely a fixed line. The last few months of observation then count for more than the org chart – see 360° feedback in consulting and professional-services firms.

The other side of the ledger: rater burden

Every nomination is a promise of time that somebody else keeps. A rater questionnaire in LEADBeyond 360° takes 10 to 15 minutes – no problem for a single request. The problem is accumulation: within a cohort, the same people are asked repeatedly – the partner who leads five managers, the senior consultant who has worked with everyone, the assistant every leader nominates.

Three countermeasures: check the overall distribution before anything is sent (who is nominated, and how often?); set a cap per leader that fits the size of the organisation – twelve is too many for 30 leaders in a firm of 150, and no problem for five leaders in a corporate division; and run a process that makes the raters' work easier: one invitation for all their requests instead of five emails, autosave, an overview of open requests, and the option to decline when the basis for observation is missing.

How LEADBeyond 360° is configured by default

As a concrete example: LEADBeyond 360° recommends, by default, nominating at least two supervisors, two peers and two direct reports. The recommendation is configurable per cycle and a soft guideline, not a gate – a leader with a single supervisor is not blocked. The anonymity threshold likewise defaults to two completed responses per group; it is set by the consulting team, not by HR. A report is generated only once the self-assessment is complete and at least two groups have reached the threshold; a consultant can deliberately override that in an individual case, and the override is logged.