How to Run a Call Centre QA Calibration Session

A calibration session is not a staff meeting with a call playing in the background. It is a measurement. You find out whether two analysts, given the same interaction and the same scorecard, produce the same score. If they do not, every coaching conversation downstream is arguing with the analyst, not with the call.

Before the session

Pick one interaction, not a montage. Choose a call people will disagree on: a borderline disclosure, a soft-skills fail, a collections call that felt rude but may have been compliant. Easy calls teach nothing.

Send the recording and the scorecard 24 hours ahead. Each analyst scores it alone, with comments, and does not discuss it. If they score together in the room, you are measuring the loudest person.

A 45-minute agenda

1. 5 minutes. Read the criteria you are testing. Not the whole form — the three lines where you already suspect drift.
2. 10 minutes. Each analyst states their score on those lines and the sentence in the call they used as evidence. No debate yet.
3. 15 minutes. Where scores differ, play that slice again. The only acceptable argument is “the criterion says X, the customer said Y.”
4. 10 minutes. Decide. Either the criterion is clear and one score was wrong, or the criterion is ambiguous and you rewrite it this week.
5. 5 minutes. Write the decision down where the next analyst will see it. A calibration that lives in someone’s notebook did not happen.

What “good” agreement looks like

You do not need 100%. You need to know the gap. Track how often two analysts pass or fail the same critical line. That is inter-rater reliability in plain language. If agreement on fatal lines is below the level your compliance team will defend, stop reporting league tables until it is fixed. A ranking built on noisy scores punishes agents for the luck of the draw.

When analysts still disagree

That is a scorecard problem, not a people problem. Split the criterion. “Showed empathy” is not scorable. “Named the impact on the customer before offering the fix” is. Fatal lines need an example of a pass and an example of a fail in the criterion help text.

Run calibration on a schedule, not after an argument. Fortnightly is enough for a stable programme. Weekly while you are changing the form.

Put the session on a scorecard everyone shares — see how QUALICORE compares, or start a free trial.