Outbound Calling with AI Agents: Appointment Reminders and Follow-Up Calls

The Case for AI Outbound Calling

Outbound calls — appointment reminders, follow-ups, payment notifications — are high-volume, repetitive tasks that are perfect for AI automation. A medical practice might need to confirm 200 appointments daily. A service company might follow up on 50 quotes. An AI agent can handle these calls 24/7, speak naturally, and adapt to each conversation, at a fraction of the cost of a human call center.

The key difference from inbound calling is that you initiate the call, which brings additional technical requirements (dialing, voicemail detection, retry logic) and legal requirements (TCPA compliance, consent management).

Building the Outbound Call Engine

Start with a scheduling system that manages the call queue and respects timing rules:

flowchart LR
    REQ(["Inbound request"])
    PII["PII detection<br/>regex plus NER"]
    POL{"Policy engine<br/>OPA or rules"}
    REDACT["Redact or mask"]
    LLM["LLM call"]
    OUT["Response"]
    AUDIT[("Append only<br/>audit log")]
    BLOCK(["Block plus<br/>notify DPO"])
    REQ --> PII --> POL
    POL -->|Allow| REDACT --> LLM --> OUT --> AUDIT
    POL -->|Deny| BLOCK
    style POL fill:#4f46e5,stroke:#4338ca,color:#fff
    style AUDIT fill:#ede9fe,stroke:#7c3aed,color:#1e1b4b
    style BLOCK fill:#dc2626,stroke:#b91c1c,color:#fff
    style OUT fill:#059669,stroke:#047857,color:#fff

from dataclasses import dataclass
from datetime import datetime, time
from enum import Enum
from typing import Optional
import asyncio

class CallStatus(Enum):
    PENDING = "pending"
    IN_PROGRESS = "in_progress"
    COMPLETED = "completed"
    VOICEMAIL = "voicemail"
    NO_ANSWER = "no_answer"
    FAILED = "failed"
    RETRY_SCHEDULED = "retry_scheduled"

@dataclass
class OutboundCall:
    id: str
    phone_number: str
    recipient_name: str
    purpose: str
    context: dict
    scheduled_at: datetime
    status: CallStatus = CallStatus.PENDING
    attempts: int = 0
    max_attempts: int = 3
    timezone: str = "America/New_York"

class OutboundCallScheduler:
    """Manages the outbound call queue with timing rules."""

    # TCPA-compliant calling hours (local time)
    EARLIEST_CALL = time(8, 0)   # 8 AM
    LATEST_CALL = time(21, 0)    # 9 PM

    def __init__(self, db_pool, twilio_client):
        self.db_pool = db_pool
        self.twilio_client = twilio_client

    async def schedule_call(self, call: OutboundCall):
        """Add a call to the queue, adjusting for calling hours."""
        from zoneinfo import ZoneInfo

        local_tz = ZoneInfo(call.timezone)
        local_time = call.scheduled_at.astimezone(local_tz).time()

        if local_time < self.EARLIEST_CALL:
            # Reschedule to 8 AM local time
            call.scheduled_at = call.scheduled_at.replace(
                hour=8, minute=0, second=0
            )
        elif local_time > self.LATEST_CALL:
            # Reschedule to 8 AM the next day
            from datetime import timedelta
            next_day = call.scheduled_at + timedelta(days=1)
            call.scheduled_at = next_day.replace(
                hour=8, minute=0, second=0
            )

        await self.persist_call(call)
        print(f"Scheduled call to {call.recipient_name} "
              f"at {call.scheduled_at}")

Initiating the Call with Personalization

When the scheduled time arrives, place the call using Twilio and pass context to your webhook:

import json
from urllib.parse import urlencode

class OutboundCallEngine:
    """Places and manages outbound AI calls."""

    def __init__(self, twilio_client, from_number, webhook_base):
        self.client = twilio_client
        self.from_number = from_number
        self.webhook_base = webhook_base

    async def place_call(self, call: OutboundCall):
        """Initiate an outbound call via Twilio."""
        # Pass context to the webhook via query params
        params = urlencode({
            "call_id": call.id,
            "recipient": call.recipient_name,
            "purpose": call.purpose,
            "context": json.dumps(call.context),
        })

        twilio_call = self.client.calls.create(
            to=call.phone_number,
            from_=self.from_number,
            url=f"{self.webhook_base}/outbound-answer?{params}",
            status_callback=f"{self.webhook_base}/outbound-status",
            machine_detection="DetectMessageEnd",
            machine_detection_timeout=10,
            timeout=30,
        )

        call.status = CallStatus.IN_PROGRESS
        call.attempts += 1
        return twilio_call.sid

The machine_detection parameter is critical — it tells Twilio to detect if a voicemail machine answered and wait for the beep before connecting your webhook. This lets your AI agent leave a coherent voicemail message.

Handling the Answered Call

Your webhook receives the call and generates a personalized conversation:

from fastapi import FastAPI, Request, Query
from fastapi.responses import Response
from twilio.twiml.voice_response import VoiceResponse

app = FastAPI()

@app.post("/outbound-answer")
async def outbound_answered(
    request: Request,
    call_id: str = Query(...),
    recipient: str = Query(...),
    purpose: str = Query(...),
    context: str = Query("{}"),
):
    form = await request.form()
    answered_by = form.get("AnsweredBy", "human")
    call_context = json.loads(context)

    response = VoiceResponse()

    if answered_by in ("machine_end_beep", "machine_end_silence"):
        # Leave a voicemail
        voicemail_msg = generate_voicemail(
            recipient, purpose, call_context
        )
        response.say(voicemail_msg, voice="Polly.Joanna")
        response.hangup()
    else:
        # Human answered — start the conversation
        greeting = generate_greeting(recipient, purpose, call_context)
        response.say(greeting, voice="Polly.Joanna")

        gather = response.gather(
            input="speech",
            action=f"/outbound-conversation?call_id={call_id}",
            speech_timeout="auto",
        )
        gather.say("Would that work for you?")
        response.say("I did not hear a response. I will call back later.")
        response.hangup()

    return Response(content=str(response), media_type="application/xml")

def generate_greeting(name, purpose, context):
    if purpose == "appointment_reminder":
        appt_date = context.get("appointment_date", "your upcoming date")
        doctor = context.get("provider_name", "your provider")
        return (
            f"Hi {name}, this is an automated reminder from "
            f"{doctor}'s office. You have an appointment scheduled "
            f"for {appt_date}."
        )
    elif purpose == "follow_up":
        return (
            f"Hi {name}, I am calling to follow up on your recent "
            f"visit. We wanted to check how you are doing."
        )
    return f"Hi {name}, this is a call from our office."

Retry Logic with Backoff

When calls go unanswered, implement intelligent retry logic:

from datetime import timedelta

async def handle_call_outcome(
    scheduler: OutboundCallScheduler,
    call: OutboundCall,
    outcome: str,
):
    """Process the call outcome and schedule retries if needed."""
    retry_delays = {
        1: timedelta(hours=2),   # First retry: 2 hours later
        2: timedelta(hours=6),   # Second retry: 6 hours later
        3: timedelta(days=1),    # Third retry: next day
    }

    if outcome in ("no-answer", "busy"):
        if call.attempts < call.max_attempts:
            delay = retry_delays.get(call.attempts, timedelta(hours=4))
            call.scheduled_at = datetime.utcnow() + delay
            call.status = CallStatus.RETRY_SCHEDULED
            await scheduler.schedule_call(call)
            print(f"Retry {call.attempts + 1} scheduled for {call.recipient_name}")
        else:
            call.status = CallStatus.FAILED
            print(f"All attempts exhausted for {call.recipient_name}")

    elif outcome == "voicemail":
        call.status = CallStatus.VOICEMAIL
        # Optionally schedule one follow-up call
        if call.attempts == 1:
            call.scheduled_at = datetime.utcnow() + timedelta(days=1)
            call.status = CallStatus.RETRY_SCHEDULED
            await scheduler.schedule_call(call)

    elif outcome == "completed":
        call.status = CallStatus.COMPLETED

TCPA Compliance Essentials

The Telephone Consumer Protection Act imposes strict rules on automated outbound calls. Non-compliance can result in fines of $500 to $1,500 per call. Key requirements include: obtain prior express consent before making automated calls, maintain an internal do-not-call list, honor opt-out requests immediately, restrict calls to 8 AM - 9 PM in the recipient's local time zone, and identify yourself at the beginning of each call.

FAQ

Do I need explicit consent for appointment reminder calls?

For healthcare appointment reminders, HIPAA and TCPA interact. You generally need prior express consent, which is typically obtained during patient registration. The consent must specifically cover automated calls. Keep records of when and how consent was obtained. For non-healthcare contexts, the rules are similar — you need prior express consent for autodialed or prerecorded calls.

How do I handle time zones correctly for calling hours?

Store each recipient's time zone in your database (derive it from their area code or address). Before placing each call, convert the current UTC time to the recipient's local time and check it against the 8 AM - 9 PM window. Use a library like zoneinfo (Python 3.9+) for accurate timezone conversions that handle daylight saving time transitions.

What is the best approach for voicemail detection?

Twilio's machine_detection with DetectMessageEnd waits for the voicemail beep before connecting, giving your AI a clean window to speak. The detection is about 90% accurate. For the 10% of cases where it misidentifies a human as a machine (or vice versa), design your greeting to work in both contexts — start with your identity and purpose, which works whether a human or voicemail system is listening.

---

#OutboundCalling #TCPACompliance #AppointmentReminders #VoiceAI #Twilio #Automation #AgenticAI #LearnAI #AIEngineering