The idea of putting a Large Language Model on a phone line sounds straightforward. Give it a system prompt describing the business, feed it the conversation transcript as it grows, stream the output to a TTS engine. In practice, production-grade conversational AI for telephony is a genuinely hard engineering problem that simultaneously touches prompt design, real-time constraints, business logic integration, error recovery, and safety. Getting any one of these wrong produces a voice agent that feels unreliable, unnatural, or outright dangerous for a business to deploy.
This post covers the full LLM integration layer for a production Voice AI system — not the research concepts, but the concrete decisions and tradeoffs that determine whether your AI agent actually performs well on real phone calls with real customers.
The System Prompt: Your Agent's Entire World
Everything about how an AI voice agent behaves — its persona, its knowledge, its constraints, its communication style — is defined by the system prompt. This is the most important artifact in a Voice AI deployment, more important than the choice of LLM model, and it requires careful engineering rather than casual authorship.
A well-structured system prompt for a voice agent defines: the agent's identity and name, the business context including services, pricing, policies and FAQs, behavioral constraints such as topics to avoid and escalation triggers, available actions like booking appointments or looking up orders, and voice-specific formatting guidelines. That last category is critical and commonly neglected. Voice prompts must produce speech-first output: short sentences, no bullet points, no markdown, no references to "the list above" or "see section 3". A response that reads beautifully as text can sound completely unnatural when a TTS engine reads it aloud.
Context Windows and Conversation Memory
A typical business phone call lasts 3 to 8 minutes. At a conversational pace of roughly 130 words per minute for both parties, a 5-minute call generates approximately 650 words of dialogue — well within the context window of even the smallest production LLMs. Longer sales calls or complex support interactions might run 20 minutes and approach 2,600 words. This still fits comfortably in an 8k-token window, though longer calls with verbose agents can start to push limits.
More interesting is cross-call memory: should the AI remember that this caller called last week and had a problem with order number 4872? That their name is Priya, they prefer speaking in Hindi, and they have been a customer for two years? This is the domain of Retrieval-Augmented Generation (RAG) combined with CRM integration. At Cirio, every call is prefixed with a dynamically assembled context block pulled from the customer's CRM history based on their phone number. The AI always sounds like it knows who it is talking to, because it actually does.
Voice AI context should be just-in-time: retrieve only what is relevant to the current call's likely purpose, not everything in the CRM. Stuffing the context with irrelevant history wastes tokens, increases LLM latency, and can actually degrade response quality by distracting the model from what matters.
Function Calling: When the AI Actually Does Things
A voice agent that can only converse but cannot take action is a very limited product. The real power comes from function calling — the ability for the LLM to emit structured JSON invocations to external systems during the conversation. Book an appointment: function call to the calendar API. Check delivery status: function call to the logistics provider API. Update a customer's address: function call to the CRM. Look up current menu availability: function call to the restaurant's inventory system.
Function calling in a real-time voice context creates latency challenges that do not exist in text applications. A calendar API might take 800 ms to respond. A CRM lookup might take 500 ms. During that time the AI must say something natural to fill the silence: "Let me check that for you, one moment..." or "I'll look that up right now." This is called latency masking, and it is essential for any voice agent that integrates with external systems. Without it, there is an unexplained silence that callers interpret as the call dropping.
Choosing the Right LLM for Voice
Model selection for voice involves a three-way tradeoff between latency, quality, and cost. Large frontier models like GPT-4o, Claude Sonnet, and Gemini Pro produce the most contextually aware and natural-sounding responses but have Time To First Token values of 300 to 800 ms under typical load, and per-token costs that add up significantly on long calls. Small fast models like Llama 3.2 3B, Gemini Flash 8B, or GPT-4o-mini have TTFT of 50 to 150 ms and dramatically lower cost but can struggle with complex multi-step instructions, domain-specific knowledge, and graceful handling of unexpected conversational directions.
A common production pattern used at Cirio is a cascade architecture: a small fast model handles simple, predictable turns — confirmations, basic FAQ answers, greeting sequences — while a larger model is invoked for complex turns involving complaint handling, multi-step scheduling, or ambiguous intent. A confidence classifier running on the transcript predicts which tier is needed before committing to either model. This approach achieves median latency close to the small model while getting large model quality where it matters.
Hallucination: The Specific Danger in Voice
LLM hallucination — generating confidently stated but factually incorrect information — is problematic in any application but particularly dangerous in voice. A user reading a chatbot response can fact-check it. A user on a phone call takes the AI's word at face value in real time. If your voice agent confidently states the wrong business hours, an incorrect price, or a fabricated policy, the consequences are real: customers show up at the wrong time, products get ordered at incorrect prices, legal disputes arise over AI-stated terms.
Hallucination mitigations for voice: constrained system prompts that explicitly instruct the agent what it knows and tell it to say "I will need to connect you with our team for that specific question" for anything outside its defined scope; RAG grounding for all factual claims (retrieve current data, never rely on the model's parametric memory for business-specific facts like prices or hours); and output validation for structured outputs like appointment times and phone numbers. The constraint here is that validation adds latency. The tradeoff between safety and speed is one of the genuine design tensions in production Voice AI.
Evaluating and Improving Your Agent
Voice AI quality evaluation is harder than text because there is no easy ground truth. Cirio uses a combination of approaches: automated transcript analysis to flag calls where the AI went off-script, said something factually incorrect, or failed to complete a stated task; human listening sessions on a sample of flagged calls each week; customer satisfaction scores correlated back to individual call sessions; and red-teaming exercises where team members try to break the agent with unusual inputs. The feedback loop from these evaluations directly drives prompt improvements and fine-tuning, making each iteration of the agent meaningfully better than the last.
