--- title: "Building a Phone Screening Agent: AI-Powered Call Screening and Routing" description: "Build an AI phone screening agent that identifies callers, detects intent, filters spam, and routes calls by priority. Covers real-time caller analysis, blocklist management, and VIP routing patterns." canonical: https://callsphere.ai/blog/building-phone-screening-agent-ai-call-screening-routing category: "Learn Agentic AI" tags: ["Call Screening", "Spam Detection", "Call Routing", "Voice AI", "Priority Routing", "Telephony"] author: "CallSphere Team" published: 2026-03-17T00:00:00.000Z updated: 2026-05-06T04:33:11.970Z --- # Building a Phone Screening Agent: AI-Powered Call Screening and Routing > Build an AI phone screening agent that identifies callers, detects intent, filters spam, and routes calls by priority. Covers real-time caller analysis, blocklist management, and VIP routing patterns. ## Why AI Call Screening Changes Everything Traditional call screening is binary — either you answer or you do not. AI screening adds intelligence: the agent answers every call, identifies the caller, understands their purpose, and makes a routing decision in seconds. Legitimate callers get connected quickly. Spam calls get blocked. High-priority calls jump the queue. This pattern is valuable for businesses that receive high call volumes (medical practices, law firms, real estate agencies) and for executives who need a smart gatekeeper without hiring a human receptionist. ## Architecture of a Screening Agent The screening agent operates in three phases: identification, intent assessment, and routing decision. Here is the core structure: ```mermaid flowchart LR CALLER(["Client"]) subgraph TEL["Telephony"] SIP["Twilio SIP and PSTN"] end subgraph BRAIN["Salon AI Agent"] STT["Streaming STT Deepgram or Whisper"] NLU{"Intent and Entity Extraction"} TOOLS["Tool Calls"] TTS["Streaming TTS ElevenLabs or Rime"] end subgraph DATA["Live Data Plane"] CRM[("CRM and Notes")] CAL[("Calendar and Schedule")] KB[("Knowledge Base and Policies")] end subgraph OUT["Outcomes"] O1(["Appointment booked"]) O2(["Reschedule completed"]) O3(["Stylist handoff"]) end CALLER --> SIP --> STT --> NLU NLU -->|Lookup| TOOLS TOOLS CRM TOOLS CAL TOOLS KB NLU --> TTS --> SIP --> CALLER NLU -->|Resolved| O1 NLU -->|Schedule| O2 NLU -->|Escalate| O3 style CALLER fill:#f1f5f9,stroke:#64748b,color:#0f172a style NLU fill:#4f46e5,stroke:#4338ca,color:#fff style O1 fill:#059669,stroke:#047857,color:#fff style O2 fill:#0ea5e9,stroke:#0369a1,color:#fff style O3 fill:#f59e0b,stroke:#d97706,color:#1f2937 ``` ```python from dataclasses import dataclass, field from enum import Enum from typing import Optional from datetime import datetime class CallerPriority(Enum): VIP = "vip" KNOWN = "known" NEW = "new" SUSPICIOUS = "suspicious" SPAM = "spam" class RoutingAction(Enum): CONNECT_IMMEDIATELY = "connect_immediately" CONNECT_WITH_ANNOUNCE = "connect_with_announce" TAKE_MESSAGE = "take_message" SEND_TO_VOICEMAIL = "send_to_voicemail" BLOCK = "block" @dataclass class ScreeningResult: caller_number: str caller_name: Optional[str] priority: CallerPriority intent_summary: str routing_action: RoutingAction confidence: float metadata: dict = field(default_factory=dict) class PhoneScreeningAgent: """AI-powered call screening and routing agent.""" def __init__(self, db_pool, ai_client, spam_checker): self.db_pool = db_pool self.ai_client = ai_client self.spam_checker = spam_checker async def screen_call(self, caller_number: str, call_sid: str): """Full screening pipeline for an incoming call.""" # Phase 1: Identification caller_info = await self.identify_caller(caller_number) # Phase 2: Spam check (fast path for known spam) if await self.spam_checker.is_spam(caller_number): return ScreeningResult( caller_number=caller_number, caller_name=None, priority=CallerPriority.SPAM, intent_summary="Spam call detected", routing_action=RoutingAction.BLOCK, confidence=0.95, ) # Phase 3: For non-spam, engage in brief conversation intent = await self.assess_intent(call_sid) # Phase 4: Make routing decision return await self.decide_routing(caller_info, intent) ``` ## Caller Identification Before the AI even speaks, look up the caller against your known contacts, CRM, and spam databases: ```python import asyncpg class CallerIdentifier: """Identifies callers from phone number lookup.""" def __init__(self, db_pool: asyncpg.Pool): self.db_pool = db_pool async def identify(self, phone_number: str) -> dict: """Look up caller in CRM and contact databases.""" # Check internal contacts first contact = await self.db_pool.fetchrow( """ SELECT name, company, relationship, priority_level, last_call_date, total_calls, notes FROM contacts WHERE phone_number = $1 AND is_active = true """, phone_number, ) if contact: return { "known": True, "name": contact["name"], "company": contact["company"], "priority": contact["priority_level"], "relationship": contact["relationship"], "call_history": { "last_call": contact["last_call_date"], "total_calls": contact["total_calls"], }, "notes": contact["notes"], } # Check recent call history for repeat unknown callers recent_calls = await self.db_pool.fetchval( """ SELECT COUNT(*) FROM call_log WHERE caller_number = $1 AND created_at > NOW() - INTERVAL '30 days' """, phone_number, ) return { "known": False, "name": None, "recent_call_count": recent_calls, } ``` ## Spam Detection Pipeline Layer multiple spam signals for accurate detection: ```python class SpamDetector: """Multi-signal spam detection for incoming calls.""" def __init__(self, db_pool, external_api_key=None): self.db_pool = db_pool self.external_api_key = external_api_key async def is_spam(self, phone_number: str) -> bool: """Check multiple spam signals.""" score = 0.0 # Check internal blocklist blocked = await self.db_pool.fetchval( "SELECT EXISTS(SELECT 1 FROM blocklist WHERE phone = $1)", phone_number, ) if blocked: return True # Check call frequency (high frequency = suspicious) calls_today = await self.db_pool.fetchval( """ SELECT COUNT(*) FROM call_log WHERE caller_number = $1 AND created_at > NOW() - INTERVAL '1 hour' """, phone_number, ) if calls_today > 5: score += 0.4 # Check against known spam patterns if self.is_spoofed_pattern(phone_number): score += 0.3 # External spam database lookup if self.external_api_key: external_score = await self.check_external_db(phone_number) score += external_score * 0.3 return score >= 0.6 def is_spoofed_pattern(self, number: str) -> bool: """Detect common spoofing patterns.""" # Numbers with all same digits, sequential patterns digits = number.replace("+", "").replace("-", "") if len(set(digits[-4:])) == 1: # Last 4 digits identical return True return False ``` ## Intent Assessment via Conversation For callers who pass the spam check, the AI engages in a brief screening conversation: ```python from openai import AsyncOpenAI class IntentAssessor: """Assesses caller intent through brief conversation.""" def __init__(self): self.client = AsyncOpenAI() async def assess(self, transcript: str, caller_info: dict) -> dict: context = "" if caller_info.get("known"): context = ( f"Known caller: {caller_info['name']} from " f"{caller_info.get('company', 'N/A')}. " f"Relationship: {caller_info.get('relationship', 'unknown')}." ) response = await self.client.chat.completions.create( model="gpt-4o-mini", messages=[ { "role": "system", "content": ( "You are a call screening assistant. Assess the " "caller's intent and urgency from their statement. " f"Context: {context}\n" "Return JSON with: intent, urgency (low/medium/high/" "emergency), summary, and recommended_action " "(connect/message/voicemail/block)." ), }, {"role": "user", "content": transcript}, ], response_format={"type": "json_object"}, temperature=0.1, ) import json return json.loads(response.choices[0].message.content) ``` ## Priority Routing Logic Combine identification and intent to make the final routing decision: ```python async def decide_routing( self, caller_info: dict, intent: dict ) -> ScreeningResult: """Determine routing based on caller identity and intent.""" # VIP callers always connect immediately if caller_info.get("priority") == "vip": return ScreeningResult( caller_number=caller_info.get("phone", ""), caller_name=caller_info.get("name"), priority=CallerPriority.VIP, intent_summary=intent.get("summary", ""), routing_action=RoutingAction.CONNECT_IMMEDIATELY, confidence=1.0, ) urgency = intent.get("urgency", "low") # Emergency calls always connect if urgency == "emergency": return ScreeningResult( caller_number=caller_info.get("phone", ""), caller_name=caller_info.get("name"), priority=CallerPriority.NEW, intent_summary=intent["summary"], routing_action=RoutingAction.CONNECT_IMMEDIATELY, confidence=0.9, ) # Known callers with medium+ urgency connect with announcement if caller_info.get("known") and urgency in ("medium", "high"): return ScreeningResult( caller_number=caller_info.get("phone", ""), caller_name=caller_info["name"], priority=CallerPriority.KNOWN, intent_summary=intent["summary"], routing_action=RoutingAction.CONNECT_WITH_ANNOUNCE, confidence=0.85, ) # Low urgency or unknown callers take a message return ScreeningResult( caller_number=caller_info.get("phone", ""), caller_name=caller_info.get("name"), priority=CallerPriority.NEW, intent_summary=intent["summary"], routing_action=RoutingAction.TAKE_MESSAGE, confidence=0.7, ) ``` ## FAQ ### How do I avoid blocking legitimate calls? Use a multi-signal approach and set conservative thresholds. Never block based on a single signal unless it is an explicit blocklist entry. Implement a "soft block" tier that sends callers to voicemail instead of disconnecting — they can still leave a message. Review blocked calls weekly and adjust thresholds. Allow users to whitelist numbers that were incorrectly flagged. ### What is the typical screening conversation duration? An effective screening interaction should complete in 10-15 seconds. The AI greets the caller, asks how it can help, and captures their initial statement. One exchange is usually enough to determine intent and urgency. Longer screening creates friction — if you cannot resolve the screening in two exchanges, route to a human. ### How do I handle the "agent whisper" when connecting a screened call? When routing a screened call to a human, use a conference bridge pattern. Connect the human first and play a whisper message (e.g., "Incoming call from John Smith regarding a billing dispute, high urgency") that only the human hears. Then bridge in the caller. Twilio supports this with the `` verb's `callerId` attribute and `` with coach mode. --- #CallScreening #SpamDetection #CallRouting #VoiceAI #PriorityRouting #Telephony #AgenticAI #LearnAI #AIEngineering --- Source: https://callsphere.ai/blog/building-phone-screening-agent-ai-call-screening-routing