Sentiment Analysis: What It Measures

Sentiment is the most quoted output of speech analytics and the one most likely to be misunderstood by everyone downstream.

What it actually does

Classifies text as positive, negative or neutral, using either lexicons of scored words, trained classifiers, or a language model.

Operates on the transcript, so it inherits every recognition error.

Produces a score per utterance, per speaker, or per call, depending on the implementation.

Some systems add acoustic features — pitch, energy, pace — which is a different signal and should be reported separately rather than blended into one number.

What it is not

It is not emotion. Emotion detection is a separate and more problematic claim, covered in its own note.

It is not customer satisfaction. A customer can be satisfied with an outcome after a call full of negative language about the problem that prompted it.

It is not a judgement of the agent. A call about a bereavement, a fraud, or a cancelled service will register negative regardless of how well it was handled. Agents on the most difficult queues score worst on sentiment, which is exactly backwards as a performance signal.

Where it fails

Negation and scope. "I wouldn't say I'm unhappy" is difficult for lexicon methods and inconsistent for classifiers.

Sarcasm. "That's brilliant, thanks" after a failure.

Domain vocabulary. Words that are negative generally and neutral in your context. "Claim", "dispute", "fault" and "charge" appear constantly in some industries and carry no emotional load there.

Transcription errors, which can invert a phrase.

Cultural and dialectal variation in how displeasure is expressed. Directness varies, and a system calibrated on one population misreads another.

Length effects. Longer calls accumulate more negative tokens, so call duration and negative sentiment correlate for mechanical reasons.

Using it responsibly

Aggregate, not individual. Sentiment across ten thousand calls in a category, trending over weeks, is genuinely informative. Sentiment on one call attached to one agent's record is not.

Normalise by queue. Compare a queue against its own baseline, never across queues. A cancellations queue and a new-sales queue are not comparable and any ranking across them ranks the queues.

Use it to find calls, not to score them. The most negative calls this week are a good review queue.

Report the change, not the level. "Sentiment in the billing category worsened this week" is a signal. "Average sentiment is 0.34" means nothing without a baseline.

Never put it in an agent scorecard. This is the single most common misuse and it produces demonstrably unfair outcomes for agents handling difficult work.

Validating it

Straightforward and rarely done.

Take two hundred calls. Have humans label them positive, negative or neutral, blind to the system's output.

Compare. Agreement in the 60 to 75 percent range is common for commercial sentiment on conversational speech.

Then check by queue and by speaker group. If agreement is materially lower for some accents or some call types, the metric is unequal and any comparison using it is confounded.

Repeat after a model update.

The honest framing

Sentiment analysis is a coarse, noisy proxy for the emotional valence of language, useful in aggregate for finding where to look, and unsuitable as a measurement of a person.

Presented that way it earns its place. Presented as "how the customer felt", it produces confident conclusions about people's inner states derived from a word-frequency calculation over an imperfect transcript.

Validating it on your own calls

The exercise that establishes whether the sentiment output means anything in your context.

Take two hundred calls, spread across queues.

Have two humans label each independently as positive, negative or neutral, blind to the system's output and to each other.

Measure human-human agreement first. If humans agree only 70 percent of the time, the system cannot be expected to exceed that, and the whole measure is coarser than the interface implies.

Then measure system-human agreement.

Break both down by queue and by speaker group. A system that agrees well on sales calls and poorly on complaints is measuring something specific to easy calls.

Report the figure alongside every sentiment output thereafter. A trend line labelled "agreement with human labelling: 68 percent" is honest and still usable.