Categorisation and Topic Detection

Categorisation assigns calls to topics — billing dispute, cancellation, technical fault, complaint. Done well it produces the operational insight that justifies the platform. Done badly it produces a chart of categories that do not mean what their names say.

How categories are built

Keyword and phrase rules. A call matches if it contains defined terms, optionally with proximity, ordering or speaker constraints. Transparent, debuggable, and dependent on someone maintaining them.

Trained classifiers. A model learns from labelled examples. Handles variation better, is harder to explain, and requires labelled data you have to produce.

Hybrid, which is what most mature deployments use: rules for the precise things, classifiers for the fuzzy ones.

Large language model classification is increasingly offered, with the properties described in a separate note: better at nuance, worse at explaining itself, and subject to change when the model is updated.

Where categorisation goes wrong

Categories defined by the vendor's template rather than by your business. Out-of-the-box category sets are generic and produce counts that do not map to anything you would act on.

Overlapping categories with no rule about precedence, so a call counted in three places inflates every number.

Categories nobody defined precisely. "Complaint" means different things to compliance, operations and marketing, and the count is quoted to all three.

Rules written once and never revised. Product names change, new issues appear, wording shifts. A category built two years ago is measuring a vocabulary that has moved.

No measurement of accuracy. The category count is reported as fact and nobody has checked how many of the matched calls actually belong.

Measuring category accuracy

The step that separates a usable deployment from a chart.

Take fifty calls matched to a category. Listen or read. How many genuinely belong? That is precision.

Take fifty calls that should belong, found another way. How many did the category catch? That is recall.

Both matter and they trade off. A tight rule has high precision and misses calls; a loose one catches everything and includes noise.

Report both alongside the count. "1,240 calls matched, precision approximately 80 percent on a fifty-call sample" is a usable figure. A bare count is not.

Re-measure quarterly and after any model or rule change.

Designing a category set

Start from decisions. What would you do differently if this category's volume doubled? If nothing, do not create it.

Mutually exclusive where possible, with a defined precedence where not.

A manageable number. Twenty well-maintained categories beat two hundred that nobody checks.

A residual category for unmatched calls, monitored. A rising unmatched proportion means the vocabulary has drifted or something new is happening, and it is the earliest signal of both.

Version the definitions with dates, so a trend across a definition change is not read as a business change.

The most valuable use

Not the pie chart of call reasons. That is what gets built and it is the least useful output.

The valuable use is change detection. A category whose volume moved this week is a question. A new phrase appearing across many calls is a problem in the field before anyone reports it.

Set up alerting on category volume, relative to its own baseline, and route it to whoever owns the process rather than to a dashboard.

A failed release, a confusing bill redesign, a supplier problem all appear in call volume within hours, and this is the mechanism that surfaces them.

Maintaining a category set

Categories decay faster than any other configuration in a speech analytics deployment.

Quarterly review of precision, on a fresh sample per category. Definitions that were accurate a year ago are not.

Watch the unmatched proportion. A rising residual means the vocabulary has drifted or something new is happening, and both are worth investigating.

Review after every product launch, campaign and policy change, because each introduces terminology the rules do not know.

Version definitions with dates, so a trend spanning a change is not read as a business event.

Retire categories nobody uses. A category set that only grows becomes unmaintainable and its precision falls across the board.

Assign an owner. Category maintenance is the ongoing work that determines whether the platform produces anything, and it is the task most often assumed to be someone else's.