Reporting Quality to Leadership
A single quality percentage is what gets asked for and it is the least informative thing you can supply. What to report instead, and how to survive the request.
Executives want one number. The one number in this field is compressed, gameable and weakly related to anything, and supplying it starts a process that ends badly.
What leadership actually needs
Are we compliant, and can we demonstrate it? The regulatory question, and the one with the largest downside.
Is the customer experience getting better or worse? From outcomes, not from QA scores.
What is broken, and is it being fixed? Process findings and their closure.
Are we exposed? Monitoring practices, data handling, automated decision-making, and whether they would survive scrutiny.
Four questions. None is answered by an average score.
The monthly page
Compliance: failure rate by requirement, from full coverage, with the trend. Exceptions listed.
Outcome: repeat contact rate and customer feedback distribution, by category, with the trend.
Process: findings raised, routed, closed, with the estimated value of the closures.
Programme health: coverage, inter-rater agreement, dispute upheld rate.
Exceptions: anything material — a new compliance gap, a model change, an accuracy finding.
One page. Anything longer is skimmed to the number, which reintroduces the problem.
Handling the request for a single score
It will come, repeatedly.
Offer the compliance rate, which is a real number with a real meaning, measured over everything.
Or offer the outcome measure, which is what the score is a proxy for.
Explain once, briefly, why the QA percentage is not it: small samples, evaluator disagreement, arbitrary weights, and it can be optimised without improving anything.
Then show the pairing: the QA score and the outcome, plotted together. If they diverge, the score is decaying. This chart does more to explain the problem than any argument.
If a single number is unavoidable, use the compliance rate and be explicit that it measures compliance rather than quality.
Reporting bad news
The programme's credibility is set by how it handles the first uncomfortable finding.
Full coverage will reveal that measured compliance is worse than the sample suggested. Say this before deploying, so the first report is not read as a decline.
A significant compliance gap goes up immediately, not in the monthly cycle.
An accuracy disparity by speaker group goes up with the analysis and the remediation, because it is a legal exposure and burying it makes it worse.
A programme that only reports good news is discovered eventually, and the discovery costs more than any individual finding.
What not to report
Agent rankings. They will be used, and the sampling and agreement problems make them indefensible.
Sentiment averages, which reflect call mix.
Evaluation counts, which measure activity.
Benchmarks against other organisations, which are not comparable because definitions differ.
Any measure without its outcome pair.
Building the annual case
What was found and fixed, with the value.
Compliance position and evidence.
The outcome trend.
What the programme cannot currently see, and what closing that would take.
That last section is what protects the budget. A programme that states its blind spots is believed about the rest, and it converts a cost line into an argument about coverage.
The single-slide version
When the monthly page is too long for the audience, this is what survives compression.
One compliance number: failure rate from full coverage, with the trend arrow and the target.
One outcome number: repeat contact rate, with the trend.
One value number: process findings closed this quarter and their estimated saving.
One risk line: anything material — a compliance gap, an accuracy finding, a monitoring exposure.
One ask, if there is one.
No QA score. If asked, explain once why the compliance rate and the outcome measure are the honest pair, and show the decay chart.
Five lines. They answer the four questions leadership actually has and they leave the detail available for anyone who wants it.