A voice agent on a telephone call does something people rarely do: it speaks and listens over the same channel at the same instant, continuously, for the whole call. That makes it unusually vulnerable to a problem telephone engineers have fought since the earliest long distance trunks. Part of what the agent says comes back on the incoming path, delayed by tens or hundreds of milliseconds, filtered by a room and a handset and a codec, and arriving as perfectly plausible speech.
A human who hears their own voice returning finds it distracting but copes. A machine does not. It transcribes the returning audio, attributes those words to the caller, and acts on them. This post explains where echo comes from, why it damages automated speech systems more than human conversations, how cancellation works, and the symptoms that identify echo quickly when a call goes wrong.
Two different echoes with the same name
Engineers use "echo" for two physically distinct phenomena, and a deployed voice system usually deals with both.
Acoustic echo is coupling between a loudspeaker and a microphone that share an environment. Sound leaves the speaker, bounces off walls, a windscreen or the caller's own face, and re-enters the microphone. Severity depends on the device: a handset held to the ear couples weakly, while a phone on speaker lying on a shop counter couples strongly, because the loudspeaker is centimetres from the microphone and the room adds reflections on top of the direct path.
Line echo, also called hybrid echo, is electrical. Traditional telephony carries the local loop on two wires, with both directions sharing the same pair, while trunks carry each direction on its own pair. The circuit converting between them is the hybrid, and it cancels the reflection only as well as its balance network matches the real impedance of the subscriber line. Lines vary with length, gauge, moisture and attached equipment, so the mismatch sends part of the far end signal straight back down the return path. Calls are largely packet based today, but the moment one touches a legacy interconnect or an analogue gateway a hybrid may still sit in the chain, and a platform that never sees the far end equipment cannot assume line echo was eliminated upstream.
Why echo is uniquely damaging for a voice agent
For an automated system, echo is not a comfort problem but a correctness problem, and it fails in several distinct ways.
- The agent transcribes itself. Returning audio is clean, fluent speech in a known language, so a recogniser has no reason to reject it. The transcript then contains the agent's own words, marked as caller input.
- False interruptions. Any logic that watches for incoming caller speech will fire on the echo. The agent stops mid sentence, having heard nobody but itself.
- Conversational feedback loops. In the worst case the agent hears its own question, treats it as a caller utterance, replies, hears that reply returning, and continues. The call degrades into a monologue no human is participating in. Even when the conversation survives, contaminated transcripts poison everything derived from them: intent labels, summaries, quality scores and training data.
How an echo canceller works
The core idea has been stable for decades. The canceller has the reference signal, meaning the audio being sent out, and the incoming signal, which contains any near end speech plus an unknown transformation of that reference. The echo path behaves approximately as a linear filter, so if the canceller can estimate that filter it can predict the echo and subtract it.
Adaptive filtering and delay
The estimate comes from an adaptive filter, usually a finite impulse response filter whose coefficients update continuously under the least mean squares rule or its normalised variant, written NLMS. The loop runs as follows:
- Pass a block of the reference signal through the current filter estimate to predict the echo.
- Subtract the prediction from the incoming signal. What remains is the error signal, which is also the outgoing audio.
- Nudge the coefficients in the direction that would have reduced that error, scaled by a step size and normalised by the reference signal energy.
Normalisation is what makes NLMS practical: without it a loud reference makes the filter jump wildly and a quiet one makes it crawl.
Two constraints govern whether this works. The filter must be long enough to span the echo tail, a few milliseconds for a handset but hundreds for a speakerphone in a tiled room. And the reference must be roughly aligned in time with the incoming signal before adaptation begins; in a packet network the round trip through encoder, jitter buffer and decoder is neither fixed nor known in advance, so delay estimation is its own subsystem. Many failures attributed to "bad cancellation" are really delay estimation failures.
The double talk problem
Adaptation works because the canceller assumes the incoming signal is mostly echo. When the near end speaker talks at the same time, that assumption breaks: the error signal now contains genuine speech the filter cannot and must not model. If adaptation continues, the coefficients are driven towards nonsense and the canceller diverges, often taking seconds to recover.
The standard remedy is a double talk detector that compares signal energies and correlations to decide when both parties are active, and freezes adaptation for the duration. The tension is real: detect double talk too eagerly and the filter stops learning, so it never tracks a changing room; detect it too late and the filter is already damaged. Conversational agents make this harder than ordinary telephony, because a caller who habitually says haan, ji or achha while the agent speaks produces double talk many times per minute.
Residual suppression and comfort noise
Linear cancellation alone rarely removes everything. What survives is residual echo, and a nonlinear post filter normally attenuates whatever still correlates with the reference. Aggressive suppression has a cost: it also attenuates the near end speaker, which is one reason some systems sound as if they clip the first syllable of a caller's reply. It removes the background noise too, producing abrupt silence, and because complete digital silence sounds like a dropped call, suppressors typically inject comfort noise shaped to match the measured background.
Nonlinearity, convergence and path changes
The adaptive filter assumes the echo path is linear. Inexpensive hardware violates that assumption enthusiastically. Small loudspeakers driven near their limit distort, amplifiers clip, plastic housings rattle, and many devices apply their own gain control or noise suppression before the microphone signal reaches the canceller, so the relationship between reference and echo shifts moment to moment. No linear filter can cancel a component that is not a linear function of the reference, which is why nonlinear post processing carries so much of the load, and why the worst echo in a deployment often appears on the cheapest handsets rather than on the noisiest lines.
This is also where learned approaches are being investigated. Deep learning based echo cancellation and residual suppression, promoted through the AEC Challenge series run alongside recent ICASSP conferences, target exactly the nonlinear distortion and difficult double talk conditions classical filters struggle with. Hybrid designs, in which a conventional filter does the bulk cancellation and a learned model cleans the residual, are a common pattern.
A canceller also does not start converged. It needs a period of reference audio to estimate the path, and the early part may leak audibly. The path is not static either: it changes when the caller moves the phone from ear to speaker, gets into a vehicle, changes hands, or when the network re-routes. Every such change forces re-convergence, so echo problems are frequently intermittent. A call that was clean for two minutes can develop echo the instant the caller switches to speakerphone.
Standards and open implementations
Two reference points are worth knowing.
ITU-T G.168 specifies requirements and test procedures for digital network echo cancellers. It defines the synthetic echo path models used for testing and sets expectations for convergence, returned echo level, tolerated echo path delay and behaviour under double talk. It describes what an acceptable canceller must achieve rather than how to build one.
WebRTC includes a well known open source acoustic echo canceller, one of the more widely studied implementations available for inspection. Its combination of linear adaptive filtering, delay estimation, double talk handling and residual suppression reflects hard won practical constraints rather than textbook idealisations.
Echo in Indian calling conditions
Deployments serving Indian consumers and businesses meet echo conditions more often than a laboratory would suggest, for reasons that are behavioural as much as technical.
- Speakerphone is a default, not an exception. Calls taken on speaker while working, driving, or so that several people in a shop can hear, are ordinary. Speaker mode maximises acoustic coupling and can be switched on mid call without warning.
- Vehicle and shop environments. A dashboard mount combines strong coupling with a reverberant cabin and road noise; a clinic reception desk adds reflections and competing speech that complicate double talk detection.
- Mixed network paths and low cost handsets. A single number may reach callers over paths that include legacy interconnects, so both hybrid and acoustic echo appear in one traffic mix, and devices with poorly matched speakers are exactly where nonlinear residual dominates.
Symptoms an engineer should recognise
Echo problems are often reported as something else. These patterns are worth memorising.
- The agent interrupts itself with nobody speaking. Almost always residual echo crossing whatever threshold triggers listening.
- Transcripts contain the agent's own phrasing as caller turns. A near certain diagnostic. Search for caller utterances that closely match the preceding agent output.
- Failures cluster on specific devices or routes. Nonlinearity or delay estimation, not a global regression.
- Echo appears only after some seconds of agent speech, or only when both parties speak together. The first suggests slow convergence or a path change; the second points at double talk handling.
- The caller's first syllable is clipped. Often over aggressive residual suppression trading near end speech for echo removal.
Keep the outgoing and incoming audio separable when reviewing a recording. If both directions are mixed into one track, distinguishing echo from genuine overlap becomes guesswork.
What this means in practice
Echo cancellation is rarely a feature anyone plans to spend time on, and it is frequently the reason a voice agent that tested well behaves erratically on real calls. The failure is silent: no component reports an error, the recogniser is confident, the dialogue logic is consistent and the transcript is coherent. It simply describes a conversation the caller was not part of.
The practical discipline is to measure rather than assume. Test with speakerphones and inexpensive handsets rather than headsets, on more than one network path, with deliberate double talk and mid call changes such as switching to speaker, and inspect transcripts for self echo rather than waiting for a complaint. At Cirio, careful attention to the unglamorous parts of telephony audio is a large part of why a conversation feels like a conversation.
Frequently Asked Questions
What causes echo on a phone call?
There are two common sources. Acoustic echo happens when sound from the loudspeaker reaches the microphone in the same room or handset, which is why speakerphones are the worst offenders. Line echo happens inside legacy telephone networks at the hybrid circuit that joins a two wire local loop to a four wire trunk, where impedance mismatch reflects part of the signal back.
Why is echo worse for a voice agent than for a human caller?
A human hears their own echo and finds it annoying but still understands the other party. A voice agent feeds the incoming audio into a recogniser, so the echo becomes text attributed to the caller. That can trigger a false interruption, corrupt the conversation transcript, or create a loop in which the agent responds to itself.
What is double talk in echo cancellation?
Double talk is when both parties speak at the same time, so the microphone signal contains both echo and genuine near end speech. An adaptive filter that keeps updating during double talk will diverge, because it tries to model the other speaker as part of the echo path. Cancellers therefore detect double talk and freeze adaptation while it lasts.
What is ITU-T G.168?
G.168 is the ITU-T recommendation that specifies performance requirements and test procedures for digital network echo cancellers, including convergence behaviour, tolerated echo path delay and stability under double talk. It is a reference point for telephony equipment rather than a description of any particular algorithm.