Interview rating scale: how many points, how to anchor them, how to calibrate

An interview rating scale is the set of numbers interviewers use to score a competency, together with a written description of what each number means. The number of points matters less than the descriptions: a 1–4 or 1–5 scale where every level is anchored to observable evidence gives more consistent ratings than a 1–10 scale labeled "poor" to "excellent". Choose a range, write behavioral anchors for each level, add a "not assessed" option and a few decision rules, then calibrate interviewers on real answers before the scale goes live.

This page is about designing the scale itself. The form it sits on is the interview scorecard template, and ready-made question-level anchors are in behavioral interview questions with a scoring rubric.

The parts of an interview rating scale

1. A range, the same for every competency. Mixing a 1–4 for one competency with a 1–5 for another makes every comparison harder.
2. Labels for each level, short enough to remember: "strong evidence", "meets the bar".
3. General anchors: what each level means for any competency.
4. Competency or question anchors: what each level sounds like for a specific competency or question.
5. A "not assessed" option, so a question nobody asked does not become a guess in the average.
6. Decision rules: how must-haves work, how interviewers' ratings combine, and whether half points are allowed.

The U.S. Office of Personnel Management's structured interview guide describes the same core: one proficiency range for all competencies, at least three levels (it suggests aiming for five to seven), labels for at least three of them, and example behaviors for each level of each competency.

1–4, 1–5, 1–7 or 1–10: choosing the number of points

What does research say? Most studies on the number of scale points come from surveys, not interviews. In one widely cited study, Preston and Colman (2000, Acta Psychologica) had 149 people rate a store or restaurant on scales with 2 to 11 points. The 2-, 3- and 4-point versions did relatively poorly on several reliability and validity indices, the indices rose with more categories up to about 7, and respondents preferred 10 points. Customers rating service are not trained interviewers rating evidence against written anchors, so read it as a caution rather than a rule: short scales throw away distinctions, and long scales only help if the extra levels are described.

A practical recommendation: use 1–4 if you are starting from nothing, 1–5 if your organization already uses five points, and put your effort into anchors either way.

Behaviorally anchored rating scales

A behaviorally anchored rating scale (BARS) replaces adjectives with examples of behavior at each level. The method is usually traced to Smith and Kendall (1963, Journal of Applied Psychology), who built rating scales anchored by examples of expected behavior. OPM's guide gives a version suited to interviews:

1. Bring together people who know the job well, such as high performers and their supervisors.
2. Each person writes, alone, how employees at each proficiency level would answer the question.
3. The group discusses the examples and keeps the ones they agree best reflect each level.
4. Interviewers use the examples as a guide rather than a script, because candidates' experiences differ.

For situational questions, OPM adds a check worth borrowing for any anchor: a separate group reads each example and says which competency it shows. Examples that are not clearly linked to one competency are dropped. An anchor that two people would file under different competencies will produce two different ratings.

The rating scale template

Copy this, pick the range, and fill in the brackets for each competency.

INTERVIEW RATING SCALE
Range: 1-4 (same for every competency)     Half points: not allowed

GENERAL ANCHORS
4  Strong evidence   Specific, first-hand examples in situations at least
                     as demanding as the role; reasoning and results
                     explained without prompting
3  Meets the bar     At least one specific, first-hand example that matches
                     the role; small gaps closed by probes
2  Below the bar     General, hypothetical, team-level ("we"), or from a much
                     simpler context; probes did not close the gap
1  No evidence       No example, or evidence of the opposite
-  Not assessed      Not asked, or ran out of time (excluded from averages)

COMPETENCY: [name]
Definition: [one sentence, agreed at intake]
4  [what a candidate at this level describes, for this role]
3  [...]
2  [...]
1  [...]

DECISION RULES
- Must-haves: a 1 is a no; a 2 needs a named follow-up before a yes
- Each interviewer submits before seeing other ratings
- Differences of 2+ points on one competency are discussed at the debrief
- Final rating per competency: [consensus after discussion / median]

If you use five points, add a level 5 above "strong evidence" for examples that exceed the role's scope, such as leading the work others in the role only take part in, and keep 3 as "meets the bar".

Anchors written for three competencies

These are competency-level anchors on a 1–4 scale, written for invented roles. Rewrite the context for your own job; keep the pattern of observable behavior at each level.

Handling customer escalations (support team lead)

Coaching direct reports (first-time people manager)

Accuracy with financial data (accounts payable specialist)

Rules for writing anchors interviewers can use

A quick test for any anchor: read it to someone who has never met the candidate and ask them to describe an answer that would earn it. If their description matches yours, the anchor works.

Decision rules that belong with the scale

• Must-haves are gates, not averages. A 4 on a nice-to-have should never cancel a 1 on a must-have.
• No half points. Ask for a choice and a sentence on what was missing for the next level.
• "Not assessed" is excluded, not scored as a 2. Divide by what was covered, or wait for the owner of that competency.
• Independent first, then discuss. OPM's guide has panel members rate independently, then compare notes and explore the basis for discrepancies before reaching a consensus rating. Decide in advance whether your final number is a consensus or a median, and write it down.
• Evidence is required. A rating without an observation behind it is returned to the interviewer.

Calibrating the scale before and after launch

Before the first candidate

1. Pilot the questions. OPM recommends a trial run of new questions with colleagues to check wording and whether they draw a range of answers.
2. Rate the same answers independently. Use two or three written or recorded answers, recorded with the participant's agreement. Each interviewer rates alone, with a reason.
3. Find the anchor behind each disagreement. Where ratings differ by two points, identify the wording that allowed both readings and rewrite it.

After the first batch of interviews

Look at each interviewer's distribution of scores. An invented example after eight candidates, four competencies each (32 ratings per interviewer):

The mean for A is (2×1 + 9×2 + 15×3 + 6×4) ÷ 32 = 89 ÷ 32 = 2.78. Interviewers may simply have met different candidates, so compare them on candidates they both saw before drawing conclusions. OPM's list of rating errors includes leniency, strictness, halo and central tendency; a distribution table is the quickest way to spot the first and last. A full session plan is in interview calibration.

Rating scale mistakes and the fix

A scale is only as consistent as the evidence each rating is based on. How to record that evidence is in evidence-based interview feedback. Interview Signal's scorecards attach quotes from the transcript to each score, which makes calibration conversations about what the candidate said rather than what each interviewer remembers.