A real-time bilingual AI phone receptionist

A commercial brokerage wanted its main number answered by an agent that could talk about specific properties in English or Spanish, understand local shorthand for places and roads, turn every call into a tracked lead, and follow up with flyers by text or email while respecting messaging consent rules.

A streaming pipeline: Twilio Media Streams into Deepgram for speech-to-text, Claude with tool use for the conversation, and ElevenLabs for streaming speech back, in μ-law end to end.

• Latency by design: speculative Claude calls on early end-of-turn (cancelled if the caller keeps talking), token-streamed speech, prompt caching and a pre-baked greeting. The design target was about 1 to 1.5 seconds to first words.
• Barge-in that works: playback is confirmed by Twilio marks, the unspoken part of a reply is computed from speech alignment, and the conversation history is rewritten to what the caller actually heard.
• Tools for the agent: search across 293 live property listings with South Florida geography (neighborhoods, highway nicknames), log every call as a CRM lead, and text or email flyers.
• Compliance in code, not in the prompt: texting refuses without a public URL, honors opt-outs and runs consent. A bug that produced phantom text sends was found and fixed.
• Around the agent: a Salesforce lead-security model (group-based edit rights, approval workflow, export lockdown), daily CRM backups to object storage, a listing audit that produced 167 WordPress listing drafts, and review pages for 35 agents that write status back to the CRM.
• Client data work operated through an AI assistant with about 140 ops scripts, and an eight-part handover runbook.

The receptionist has answered the brokerage's main number since April 2026, with text messaging added after carrier approval in June. It was handed over with runbooks in mid-2026.

I have no call-volume, lead or time-saved figures, so none are claimed. The latency figure above is a design target, not an independent measurement.