What Speech Analytics Actually Does

Speech analytics is a pipeline with four stages, and knowing which stage a claim refers to explains most of the confusion about what these systems can do.

The pipeline

One: audio capture. Recording, ideally with each speaker on a separate channel. Where both speakers share one channel, everything downstream degrades.

Two: automatic speech recognition. Audio to text. This is where the errors that propagate through everything else are introduced.

Three: analysis over the transcript. Search, categorisation, classification, statistical measures.

Four: acoustic analysis, separately from the words. Silence, talk-over, pace, volume.

Almost everything a vendor calls insight happens at stage three, operating on text produced at stage two, whose accuracy determines the ceiling.

What it does well

Finding things. Every call mentioning a competitor, a regulator, a specific product, a phrase that indicates escalation risk. This is search over a complete corpus and it is the strongest capability by a wide margin.

Counting. How many calls were about a particular topic, this week compared to last. Trend detection over full coverage is genuinely valuable and impossible manually.

Checking presence. Was the required disclosure said? Binary, verifiable, and the clearest compliance application.

Acoustic measures. Silence duration, talk-over frequency, longest monologue. These are measured rather than inferred and are among the most reliable outputs.

Aggregation. Surfacing that a hundred calls this week mentioned a specific error message is a process finding no manual sample would produce.

What it does poorly

Judging quality. Whether the agent gave correct advice, whether the resolution was appropriate, whether the customer was treated well. These require understanding the situation.

Sentiment, which is covered separately because the gap between what it measures and what it is presented as measuring is large.

Emotion, which has a scientific problem discussed in its own note.

Anything depending on words the recogniser got wrong. Product names, account numbers, technical terms and proper nouns are exactly where accuracy is worst and where the analysis most often depends on them.

Sarcasm, implication and context. "That's great" is not always positive.

Keyword spotting versus transcription

Two architectures with different properties, and vendors are not always clear about which they use.

Phonetic keyword spotting searches the audio for sound patterns without producing a full transcript. Fast, robust to accent variation in some implementations, and limited to terms you specified in advance.

Full transcription produces text you can then search, classify and analyse in ways nobody planned for. Slower, more expensive, and the basis of everything modern.

The practical difference: with a transcript you can ask a new question of last year's calls. With keyword spotting you can only ask the questions you configured before the calls happened.

What the output is worth

As a detection system: high value. Finding the twelve calls out of forty thousand where something specific happened is not otherwise possible.

As a measurement system: moderate, with the accuracy caveat.

As a judgement system: low, and this is what most of the marketing describes.

The productive framing is that speech analytics tells you where to look, and a person decides what it means. Programmes built on that division work. Programmes that automate the judgement produce scores agents correctly dispute.

The dependency nobody plans for

Everything above rests on transcription accuracy, which varies by speaker, by accent, by audio quality and by vocabulary.

Nobody measures it, in most deployments. The vendor quotes a figure from their own testing and the buyer never checks it against their own calls.

Measuring it is straightforward — transcribe a sample, correct it by hand, count the errors — and it is the single most useful thing a new deployment can do, because it establishes what the ceiling actually is before anyone builds reporting on top of it.

A first month that produces something

Deployments frequently spend the first months configuring dashboards. A more productive opening.

Week one: measure transcription accuracy on thirty of your own calls, stratified by accent and queue. This sets the ceiling for everything else.

Week two: build three categories from problems you already know about, and measure their precision on fifty calls each.

Week three: run the acoustic measures across everything and look at silence by queue and by system. This usually produces a process finding immediately and it does not depend on transcription at all.

Week four: take the strongest finding to the owner with the frequency, the estimated cost and three recordings.

One fixed process problem in the first month establishes what the platform is for, and it does more for the programme's future than any dashboard.