Skip to main content

Call Agent

Audience: Sales · Management · QA · Support · Engineering · Where in app: Settings → AI Agents → Call Agent (voice, personality, call behavior, routing); Settings → Calls → Preferences · Plan availability: All plans for web/inbound voice; outbound campaigns are admin-gated (ai_outbound_call_allowed). Usage bounded by AI Credits — see AI Credits

The Call Agent is the AI voice that handles audio calls — when a visitor clicks the audio-call button in the widget, when someone phones a provisioned AI number, or when an outbound campaign places a call. It speaks with a natural text-to-speech voice, listens, answers from your Knowledge Base, and can perform actions mid-call (book a meeting, look up a product, transfer to a human) using server-side tools.

What it does​

  • Answers inbound phone calls on a provisioned AI number.
  • Answers browser/web voice calls started from the widget.
  • Places outbound calls (campaigns, proactive callbacks, follow-ups).
  • Holds a natural back-and-forth spoken conversation, grounded in the KB via RAG.
  • Calls server-side tools/functions mid-conversation: book a meeting, search products/KB, transfer to a human, send a follow-up email, update a CRM contact.
  • Verifies caller identity for inbound (caller-ID match) when enabled.
  • Captures a transcript, quality score, sentiment, outcome, and termination reason after every call, and updates the identity graph with extracted facts.

How it works​

Voice providers. Three voice stacks are supported, selected via companies.ai_config.call_model_provider:

ProviderNotes
ElevenLabs Conversational AIPrimary stack. High-quality voices, configurable agents, mature tool-use, native phone numbers. Inner LLM (the "brain") is per-agent selectable from a curated agent_type: 'call' list (ai-model-pricing.controller.ts: claude-sonnet-4-5 default, GPT-4.1/5-mini/5.1/5.2, Gemini rows); unsupported models silently fall back to ELEVENLABS_INNER_LLM (default claude-sonnet-4-5). A Baseten-hosted custom LLM (zai-org/GLM-4.7) can back the agent instead — its tokens are priced in cost-tracking but recorded at credits = 0 (absorbed platform cost, never billed to tenants).
Ultravox (fixie-ai/ultravox)Low-latency, single-model architecture, custom tools.
OpenAI RealtimeDirect OpenAI integration, audio-native model.

Tool / function calling. During a call the agent can invoke server-side functions exposed by ms-ai's knox-voice/ module (knox-voice-tools.service.ts). Typical tools: book a meeting (Google/Calendly/GHL), look up a product (RAG over the product catalog), search the company knowledge base, transfer/take over to a human, send a follow-up email, update a CRM contact. Tool schemas are converted to the provider's format (zod-to-elevenlabs-schema.ts) and the calls flow provider → WebSocket → ms-ai → ms-communication. The public calendar tool endpoints are POST /tools/calendar/.../availability and .../booking (rate-limited).

RAG grounding. Same retrieval path as the Chat Agent: the question is embedded, top candidates are pulled from the company embeddings pgvector table and Cohere-reranked, and injected as context. Product lookups run over the product catalog (rag_products). See Knowledge Base.

Multilingual. ElevenLabs language_detection lets the agent auto-switch mid-call into any of ~70 preset languages (MULTILINGUAL_CODES in ms-ai/src/elevenlabs/elevenlabs.service.ts, built into languagePresets on every agent sync). The configured agent_language still governs the greeting.

Configuration & options​

Voice selection & persona​

Choose the voice (gender, tone, accent/style) and preview it before going live. The voice is stored as agent_voice_id (ElevenLabs voice ID); language as agent_language (BCP-47 code).

Outbound calling — companies.ai_calling​

FieldWhat
enableMaster switch.
settingquick or advanced.
agent_instructionsSystem prompt (advanced).
agent_nameVoice persona name.
tonePersona tone.
response_sizeconcise / detailed.
greeting_messageOpening line.

Inbound calling — companies.ai_inbound_call​

FieldWhat
enableMaster switch — provisions an inbound number routed to the AI.
model / model_providerLLM + provider.
agent_voice_idElevenLabs voice ID.
agent_languageBCP-47 language code.

Caller-ID may verify the visitor's identity if identity_graph_config.accept_inbound_call_verification is on.

Outbound campaigns (multi-agent)​

companies.ai_outbound_call_enabled: true   // requires admin gate ai_outbound_call_allowed
companies.ai_outbound_call_agents: [
  { name, prompt, voice_id, model_provider, model, webhook_secret, ... },
  ...
]

Each entry is a distinct persona/script. Launched from the dashboard's outbound campaign UI, or programmatically by Nox via knox-place-call (when enabled).

Call behavior & routing​

  • Call routing — companies.ai_config.call_routing: ai (AI handles the call) / ai_to_human (AI as the first leg, then handoff) / human.
  • Duration limits / silence handling / hold behavior — cap call length to manage credits; configure re-prompts on silence ("Are you still there?").
  • Provider data models (ms-ai): elabs_agents (ElevenLabs agent config), elabs_phone_numbers (mapped inbound numbers).

Behaviors & edge cases​

  • Transport. Widget AI calls now run over WebRTC (LiveKit) — ms-communication hardcodes calling_type: 'webrtc', ms-ai returns a LiveKit conversation token, and the widget uses the ElevenLabs SDK callbacks for transcripts/messages/tools (the websocket signed-URL path remains as fallback). PSTN calls run through the carrier bridges (Twilio + Telnyx) — see Telephony.
  • Takeover. A human agent joins and the AI hands off; audio routes through Twilio WebRTC. Visibility controlled by companies.communication_options.take_over_permissions.
  • Monitor. A human can listen silently — monitor_call socket event, audio streamed via audio_chunk (on WebRTC calls the widget taps the agent's LiveKit audio track to feed the stream).
  • Recording. ElevenLabs/Ultravox transcripts are always captured; audio recording is optional per provider settings. companies.always_twilio_calls = true forces the Twilio path even for AI calls (useful for compliance environments that require Twilio's recording).
  • Post-call enrichment. After the call: transcript saved to knox_call_transcripts; heuristic quality score computed; high-stakes calls re-scored by an LLM-judge (quality_method: llm_judge); sentiment, outcome, topics, and termination reason populated; booking confirmations sent; identity graph updated with extracted facts. PSTN calls additionally get call_sentiment + call_outcome classification, voicemail detection, and a signed recording_url in the post-call webhook — see Telephony › Post-call pipeline.
  • Credit exhaustion. Voice consumes credits per minute (faster than chat). Inbound/web falls back to a human if available, otherwise a missed-call. Outbound campaigns pause until refilled. See AI Credits.

Quality scoring​

knox_call_transcripts.quality_score (0..1) with quality_method:

MethodWhen
heuristicFast — based on duration, turn count, agent-failed-to-answer signals.
llm_judgeSlower — LLM scores the transcript across multiple dimensions (high-stakes calls).
skippedToo short to score meaningfully.

quality_breakdown is available via knox-aggregate. Nox can report: which provider is configured, the agent's prompt, call/booking counts, quality breakdown, top termination reasons, and whether outbound is enabled. Tools: knox-call-transcripts, knox-call-transcript-full, knox-recent-bookings, knox-aggregate(metric=quality_breakdown).

Plan & limits​

  • Web and inbound voice are on all plans; outbound campaigns require the ai_outbound_call_allowed admin gate.
  • Voice usage is bounded by AI Credits, billed per minute (provider voice cost) plus the inner LLM. ElevenLabs voice is priced as a credit-based plan rate; the bundled inner LLM is a LLM_PASSTHROUGH charge on top. See AI Credits.
  • Voice usage is tracked by location (voice_inbound / voice_outbound / voice_web) so management can see, e.g., whether browser voice is costing more than phone calls.
  • Verify per-tenant phone-number provisioning and any region/compliance constraints.

Technical implementation​

  • Service: ms-ai, modules knox-voice/ (KnoxVoiceModule — orchestration, tool bridge, context/registry stores, post-call service) and elevenlabs/ (telephony: start calls, webhooks, phone numbers).
  • Endpoints: POST /elevenlabs/start-audio-call (returns a LiveKit token for calling_type: 'webrtc', signedUrl for websocket), GET /elevenlabs/voices, POST /elevenlabs/create-tools, POST /elevenlabs/webhook/post-call (HMAC-verified elevenlabs auth), number endpoints (import-phone-number/{twilio,telnyx}, buy-phone-number/twilio/*, GET /elevenlabs/phone-numbers/:company_id — see Telephony), and calendar tools POST /tools/calendar/.../availability · .../booking. See ms-ai/API_ENDPOINTS.md.
  • Tool schema bridge: knox-voice/zod-to-elevenlabs-schema.ts; tool impls in knox-voice/knox-voice-tools.service.ts.
  • Prompts: knox-voice/knox-voice.prompt.ts, knox-voice-brief.prompt.ts.
  • Post-call: knox-voice/knox-voice-post-call.service.ts; transcripts in knox_call_transcripts.
  • Inner LLM: ELEVENLABS_INNER_LLM env (default claude-sonnet-4-5) in elevenlabs/elevenlabs.service.ts.
  • Widget: the browser audio-call flow lives in the widget (aiCall.ts / ChatWidget) → ms-communication → ms-ai start-audio-call; LiveKit transport with a 30s token wait and a ringback loop (on-call-ringing.mp3) during connect.
  • Cost: cost-tracking/ books per-minute usage_charge (provider) + LLM_PASSTHROUGH for the inner LLM. See AI Credits.