A real-time AI voice-intake agent built on Twilio ConversationRelay — streaming LLM turns over a WebSocket, a live-editable personality on top of a compliance core locked in code, bilingual mid-call switching, and a transcript-only posture with zero call recording.
GrowthOS onboards new customers through a structured intake — business name, social handles, audience, tone, competitors, call-to-action, booking link. Two channels already existed: a web form and a chat agent, both merging into one canonical intake record. The team wanted a third: a phone call. An agent that dials the prospect, holds a natural conversation, and walks out with the same structured data.
A phone call is the most intimate channel a product can use. When the voice on the other end sounds human, people extend it human trust — which is exactly why regulators moved on it first. The California B.O.T. Act makes undisclosed bot communication with intent to mislead unlawful. Article 50 of the EU AI Act requires that people interacting with an AI system be informed of it. The FTC has been explicit that an AI pretending to be a person is a deception problem, not a style choice.
Most teams "solve" this with a line in the system prompt: always say you're an AI. That is not compliance — that is a suggestion to a language model. A prompt can be overridden by a later instruction, edited by an operator who doesn't know the law, or simply ignored by the model on a bad turn. If the disclosure can be removed by typing into a textbox, it is not a control — it is a default.
The brief I set for myself: build a voice agent whose personality, greeting, and voice are fully operator-editable from an admin panel — live, no redeploy — while the honesty guarantees are structurally impossible to edit away, because they live in code paths, not prompt text.
Twilio ConversationRelay turns a live phone call into a WebSocket session: Twilio handles telephony, speech-to-text, and text-to-speech at the carrier edge, and streams the conversation as text frames to a Node.js relay server that owns the brain.
The relay server is the only component I own end to end — and it is deliberately thin. It attaches its WebSocket message listener synchronously and queues frames while it initializes, because Twilio's setup frame arrives immediately and an early version was dropping it during database lookups, producing dead-air calls. Each turn, the caller's transcribed speech arrives as a text frame; the server appends it to the conversation history, calls Gemini with streaming enabled, buffers the reply, strips silent control tags, and sends the clean text back over the socket for Twilio to synthesize.
The agent's "brain" is not baked into the deploy. Personality, greeting wording (EN and ES), model choice, temperature, voice selection, speaking rate, enabled languages, off-topic handling, and closing wording all live in a NocoDB config row read at call start with a 30-second cache — an operator edits the admin panel and the very next call behaves differently, no redeploy. Every value is bounded and validated server-side: unknown models and voices fall back to defaults, out-of-range numbers are clamped, and voices come from a server-side allowlist so a garbage value can never reach a live call. Every config save writes an append-only audit row with actor and before/after state.
Intake questions come from the same admin-editable question registry that drives the web form and the chat agent — one vocabulary, three channels. On hang-up, the relay posts the transcript and extracted answers to the n8n intake webhook, where they merge into the same canonical onboarding record as the other two channels.
The operator can rewrite every word the agent says — its name, its personality, its greeting, its sign-off. The one thing no admin textbox can produce is an agent that hides being an AI. That guarantee is enforced in three code layers, aligned with the California B.O.T. Act, EU AI Act Article 50, and FTC guidance on deceptive AI.
ensureAiIdentity()The first thing spoken on every call, before any data is collected, discloses that the caller is talking to an AI. The wording is the operator's — casual, branded, short, their call — but the code inspects the configured opener, and if it contains no AI mention in either language, a compact "I'm an AI" clause is prepended automatically. The operator controls every word except the one that matters legally.
If anything in the pipeline — an operator's custom wording, a model hallucination, a prompt-injection attempt by the caller — would produce a claim of being a real person ("I'm not a bot", "no soy un bot", "I'm human"), a denial guard detects it and fails closed to the compliant default. Asked "are you human?", the agent answers truthfully, every time, regardless of what any editable text says. The failure mode is honesty, not silence and not a lie.
filterPersonality()The editable personality text is sanitized server-side before it is injected into the prompt. Any line attempting to undo the disclosure — "never say you're an AI", "pretend to be human", and their variants — is stripped before it can reach a call, and an explicit precedence clause makes the locked compliance core win any conflict with operator text. The admin panel is powerful; it is simply not powerful enough to create a liar.
Why code and not prompts? Because the threat model includes the operator. An admin under sales pressure might reasonably think a "more human" agent converts better and edit the greeting accordingly — not maliciously, just uninformed. With a prompt-level disclosure, that edit silently creates legal exposure under statutes with real teeth. With the honesty floor, the same edit produces a perfectly reworded greeting that still discloses, because the disclosure was never in the editable text to begin with.
The same governance-first posture killed call recording. An early build auto-recorded every call; the data-governance review flagged two-party-consent exposure and it was removed entirely. The pipeline is transcript-only by design — the structured intake answers and conversation text are kept, the audio never exists. You cannot leak, subpoena, or mishandle a recording that was never made.
Compliance makes the agent legal. These are the decisions that make it good — resumable, bilingual, fast, and isolated from its sibling agents.
Phone calls drop. Most voice agents restart the whole script, which is exactly how you teach a prospect to stop answering. At call start the relay fetches the caller's own intake row by exact onboarding-token match and injects only registry-defined answers into the system prompt as already on file — questions answered on a previous call, in the web form, or in chat are removed from the ask-list, so the conversation resumes at the first missing item. It also captures answers mid-call: the model emits a hidden FIELDS line every turn, extracted server-side, so even a call that drops mid-sentence keeps everything said up to that point. The lookup requires exactly one match, degrades gracefully to stateless behavior on failure, and seeded values are never logged.
The opener offers English or Spanish. When the model decides to switch, it emits a silent [LANG] control tag; the server intercepts it and sends a ConversationRelay language message that flips both the transcription language and the TTS voice in one move — English Journey voice to Spanish Neural2 mid-call. Replies are buffered rather than streamed to the socket precisely so control tags can never leak into spoken audio.
Calls were intermittently answering after 5–6 seconds of silence — an eternity on the phone. Root cause: the voice model was running with "thinking" mode on, which spiked per-turn latency and could eat the token budget into an empty reply. Conversational turns don't need extended reasoning — the hard rules in the prompt carry it — so thinking was disabled for voice turns. Replies now land consistently around one second.
The chat agent and the voice agent each read their own config row, keyed by agent, created if missing. That wasn't the first design: an earlier iteration pointed the voice agent at a shared config row — which meant saving the voice agent's settings would have silently overwritten the chat agent's brain. One agent's admin edit clobbering another agent's live behavior is the kind of bug you only get to ship once; the row-per-agent contract makes it structurally impossible.
Disclosure requirements vary by jurisdiction, so the floor itself became configurable — carefully. It ships ON by default, a fresh config always discloses, and only an explicit opt-out relaxes it. Flipping that switch is an audited event: every save writes an append-only log row with actor and before/after state, so the operator formally owns the decision for their region. Compliance by default, flexibility on the record.
I build conversational AI where the compliance guarantees live in code, not in a prompt someone can edit away. If your agent talks to real people, the honesty floor is not optional.
Let's Work Together