Build a Customer Support Agent from Scratch: Python, OpenAI, and Twilio in 60 Minutes

Why Build a Customer Support Agent?

Customer support is one of the highest-ROI use cases for AI agents. Unlike simple chatbots that follow rigid decision trees, an agentic customer support system can reason about the customer's problem, look up real data, take actions in backend systems, and escalate to humans when necessary. In this tutorial, you will build a fully functional customer support agent in under 60 minutes.

The agent you build will handle voice calls through Twilio, reason about customer problems using OpenAI's Agents SDK, and interact with your backend through five purpose-built tools. By the end, you will have a working system that can look up customers, check order status, create support tickets, transfer calls, and answer frequently asked questions.

Architecture Overview

The system consists of three layers:

flowchart LR
    CORPUS[("Pre-training corpus<br/>trillions of tokens")]
    FILTER["Quality filter and<br/>dedupe"]
    TOK["BPE tokenizer"]
    SHARD["Shard plus<br/>data parallel"]
    GPU{"GPU cluster<br/>FSDP or DeepSpeed"}
    CKPT[("Checkpoints<br/>every N steps")]
    LOSS["Loss curve plus<br/>eval gates"]
    SFT["SFT phase"]
    DPO["DPO or RLHF"]
    BASE([Base model])
    INSTR([Instruct model])
    CORPUS --> FILTER --> TOK --> SHARD --> GPU
    GPU --> CKPT --> LOSS
    LOSS --> BASE --> SFT --> DPO --> INSTR
    style GPU fill:#4f46e5,stroke:#4338ca,color:#fff
    style LOSS fill:#f59e0b,stroke:#d97706,color:#1f2937
    style INSTR fill:#059669,stroke:#047857,color:#fff

1. Telephony Layer — Twilio handles incoming calls and converts speech to text
2. Agent Layer — OpenAI Agents SDK processes the transcribed speech, reasons about what to do, and calls tools
3. Backend Layer — FastAPI serves as the tool execution engine, connecting to your database and ticketing system

┌──────────────┐     ┌───────────────────┐     ┌──────────────┐
│   Customer   │────▶│   Twilio Voice     │────▶│  FastAPI      │
│   (Phone)    │◀────│   + STT/TTS        │◀────│  + Agent SDK  │
└──────────────┘     └───────────────────┘     └──────┬───────┘
                                                       │
                                          ┌────────────┼────────────┐
                                          ▼            ▼            ▼
                                     PostgreSQL   Ticketing     FAQ Store

Prerequisites

Before starting, make sure you have:

• Python 3.11+ installed
• A Twilio account with a phone number
• An OpenAI API key with Agents SDK access
• PostgreSQL running locally or remotely
• ngrok or a public URL for Twilio webhooks

Step 1: Project Setup and Dependencies

Create a new project and install dependencies:

mkdir support-agent && cd support-agent
python -m venv venv && source venv/bin/activate

pip install fastapi uvicorn openai-agents twilio psycopg2-binary pydantic python-dotenv

Create the project structure:

mkdir -p app/{tools,models,services}
touch app/__init__.py app/main.py app/agent.py
touch app/tools/__init__.py app/tools/customer.py app/tools/orders.py
touch app/tools/tickets.py app/tools/transfer.py app/tools/faq.py
touch .env

Set up your environment variables in .env:

OPENAI_API_KEY=sk-proj-your-key-here
TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
TWILIO_AUTH_TOKEN=your-auth-token
DATABASE_URL=postgresql://user:pass@localhost:5432/support_db

Step 2: Define the Database Models

Create a simple schema for customers and orders:

# app/models/database.py
import psycopg2
from psycopg2.extras import RealDictCursor
from functools import lru_cache
import os

def get_connection():
    return psycopg2.connect(
        os.getenv("DATABASE_URL"),
        cursor_factory=RealDictCursor
    )

def init_db():
    conn = get_connection()
    cur = conn.cursor()
    cur.execute("""
        CREATE TABLE IF NOT EXISTS customers (
            id SERIAL PRIMARY KEY,
            phone VARCHAR(20) UNIQUE NOT NULL,
            name VARCHAR(100) NOT NULL,
            email VARCHAR(200),
            tier VARCHAR(20) DEFAULT 'standard',
            created_at TIMESTAMP DEFAULT NOW()
        );
        CREATE TABLE IF NOT EXISTS orders (
            id SERIAL PRIMARY KEY,
            customer_id INTEGER REFERENCES customers(id),
            order_number VARCHAR(50) UNIQUE NOT NULL,
            status VARCHAR(30) DEFAULT 'pending',
            total DECIMAL(10,2),
            items JSONB,
            created_at TIMESTAMP DEFAULT NOW(),
            updated_at TIMESTAMP DEFAULT NOW()
        );
        CREATE TABLE IF NOT EXISTS tickets (
            id SERIAL PRIMARY KEY,
            customer_id INTEGER REFERENCES customers(id),
            subject VARCHAR(200) NOT NULL,
            description TEXT,
            priority VARCHAR(20) DEFAULT 'medium',
            status VARCHAR(20) DEFAULT 'open',
            created_at TIMESTAMP DEFAULT NOW()
        );
    """)
    conn.commit()
    cur.close()
    conn.close()

Step 3: Build the Five Agent Tools

Each tool is a Python function decorated with the Agents SDK tool decorator. The agent decides which tools to call based on the conversation context.

Tool 1: Customer Lookup

# app/tools/customer.py
from agents import function_tool
from app.models.database import get_connection

@function_tool
def lookup_customer(phone_number: str) -> str:
    """Look up a customer by their phone number. Returns customer name,
    email, tier, and account ID. Use this when the caller needs to be
    identified or when you need their account details."""
    conn = get_connection()
    cur = conn.cursor()
    cur.execute(
        "SELECT id, name, email, tier FROM customers WHERE phone = %s",
        (phone_number,)
    )
    row = cur.fetchone()
    cur.close()
    conn.close()

    if not row:
        return "No customer found with this phone number. Ask for their email or name to search further."

    return (
        f"Customer found: {row['name']} (ID: {row['id']}), "
        f"Email: {row['email']}, Tier: {row['tier']}"
    )

Tool 2: Order Status Check

# app/tools/orders.py
from agents import function_tool
from app.models.database import get_connection

@function_tool
def check_order_status(order_number: str) -> str:
    """Check the status of an order by order number. Returns order status,
    items, total, and timestamps. Use when a customer asks about their order."""
    conn = get_connection()
    cur = conn.cursor()
    cur.execute(
        """SELECT o.order_number, o.status, o.total, o.items,
                  o.created_at, o.updated_at, c.name as customer_name
           FROM orders o JOIN customers c ON o.customer_id = c.id
           WHERE o.order_number = %s""",
        (order_number,)
    )
    row = cur.fetchone()
    cur.close()
    conn.close()

    if not row:
        return f"No order found with number {order_number}. Ask the customer to verify the order number."

    return (
        f"Order {row['order_number']} for {row['customer_name']}: "
        f"Status: {row['status']}, Total: ${row['total']}, "
        f"Items: {row['items']}, "
        f"Placed: {row['created_at']}, Last updated: {row['updated_at']}"
    )

Tool 3: Create Support Ticket

# app/tools/tickets.py
from agents import function_tool
from app.models.database import get_connection

@function_tool
def create_ticket(
    customer_id: int,
    subject: str,
    description: str,
    priority: str = "medium"
) -> str:
    """Create a new support ticket for a customer. Use when the issue
    cannot be resolved immediately and needs follow-up. Priority can be
    low, medium, high, or urgent."""
    if priority not in ("low", "medium", "high", "urgent"):
        priority = "medium"

    conn = get_connection()
    cur = conn.cursor()
    cur.execute(
        """INSERT INTO tickets (customer_id, subject, description, priority)
           VALUES (%s, %s, %s, %s) RETURNING id""",
        (customer_id, subject, description, priority)
    )
    ticket_id = cur.fetchone()["id"]
    conn.commit()
    cur.close()
    conn.close()

    return f"Ticket #{ticket_id} created successfully with {priority} priority. The customer will receive an email confirmation."

Tool 4: Transfer Call

# app/tools/transfer.py
from agents import function_tool

@function_tool
def transfer_to_human(
    department: str,
    reason: str
) -> str:
    """Transfer the call to a human agent in the specified department.
    Departments: billing, technical, returns, management. Use this when
    the customer explicitly requests a human or when the issue is too
    complex for automated resolution."""
    valid_departments = {
        "billing": "+15551001001",
        "technical": "+15551001002",
        "returns": "+15551001003",
        "management": "+15551001004",
    }
    target = valid_departments.get(department.lower())
    if not target:
        return f"Unknown department '{department}'. Available: {', '.join(valid_departments.keys())}"

    return f"TRANSFER_SIGNAL::{target}::Transferring to {department}. Reason: {reason}"

Tool 5: FAQ Search

# app/tools/faq.py
from agents import function_tool

FAQ_DATABASE = {
    "return_policy": "Items can be returned within 30 days of delivery. Items must be in original packaging. Refunds are processed within 5-7 business days.",
    "shipping_times": "Standard shipping: 5-7 business days. Express: 2-3 business days. Overnight: next business day. Free shipping on orders over $50.",
    "payment_methods": "We accept Visa, Mastercard, American Express, PayPal, Apple Pay, and Google Pay.",
    "warranty": "All products come with a 1-year manufacturer warranty. Extended warranties are available for purchase at checkout.",
    "hours": "Customer support is available Monday through Friday 8am to 8pm EST, and Saturday 9am to 5pm EST.",
}

@function_tool
def search_faq(query: str) -> str:
    """Search the FAQ database for answers to common questions. Use this
    for general policy questions before creating tickets."""
    query_lower = query.lower()
    results = []
    for key, answer in FAQ_DATABASE.items():
        if any(word in query_lower for word in key.split("_")):
            results.append(f"**{key.replace('_', ' ').title()}**: {answer}")

    if not results:
        return "No FAQ matches found. You may need to create a ticket for this question."
    return "\n".join(results)

Step 4: Create the Agent

Wire all five tools together into a single agent with clear instructions:

# app/agent.py
from agents import Agent
from app.tools.customer import lookup_customer
from app.tools.orders import check_order_status
from app.tools.tickets import create_ticket
from app.tools.transfer import transfer_to_human
from app.tools.faq import search_faq

support_agent = Agent(
    name="Customer Support Agent",
    instructions="""You are a helpful customer support agent for an e-commerce company.

    RULES:
    1. Always greet the customer warmly and identify them by looking up their phone number.
    2. Listen carefully to their issue before taking action.
    3. Use the FAQ tool first for policy questions before escalating.
    4. Only create tickets for issues that need follow-up.
    5. Transfer to a human if the customer requests it or if you cannot resolve the issue after 2 attempts.
    6. Always confirm actions before executing them.
    7. Keep responses concise and conversational — this is a phone call.
    8. Never reveal internal system details or tool names to the customer.
    """,
    tools=[
        lookup_customer,
        check_order_status,
        create_ticket,
        transfer_to_human,
        search_faq,
    ],
    model="gpt-4o",
)

Step 5: Build the FastAPI Server with Twilio Integration

The server handles incoming Twilio webhooks and routes them through the agent:

# app/main.py
import os
from contextlib import asynccontextmanager
from fastapi import FastAPI, Request, Response
from twilio.twiml.voice_response import VoiceResponse, Gather
from agents import Runner
from app.agent import support_agent
from app.models.database import init_db
from dotenv import load_dotenv

load_dotenv()

@asynccontextmanager
async def lifespan(app: FastAPI):
    init_db()
    yield

app = FastAPI(lifespan=lifespan)

# In-memory session store (use Redis in production)
sessions: dict[str, list[dict]] = {}

@app.post("/voice/incoming")
async def handle_incoming_call(request: Request):
    """Handle initial incoming call from Twilio."""
    form = await request.form()
    caller = form.get("From", "unknown")

    sessions[caller] = []

    response = VoiceResponse()
    response.say("Welcome to customer support. How can I help you today?")
    gather = Gather(
        input="speech",
        action="/voice/process",
        speech_timeout="auto",
        language="en-US",
    )
    response.append(gather)
    return Response(content=str(response), media_type="application/xml")

@app.post("/voice/process")
async def process_speech(request: Request):
    """Process speech input and run through the agent."""
    form = await request.form()
    caller = form.get("From", "unknown")
    speech_result = form.get("SpeechResult", "")

    if not speech_result:
        response = VoiceResponse()
        response.say("I did not catch that. Could you please repeat?")
        gather = Gather(
            input="speech",
            action="/voice/process",
            speech_timeout="auto",
        )
        response.append(gather)
        return Response(content=str(response), media_type="application/xml")

    # Build conversation history
    history = sessions.get(caller, [])
    history.append({"role": "user", "content": speech_result})

    # Add caller context
    context_msg = f"The caller's phone number is {caller}."
    messages = [{"role": "user", "content": context_msg}] + history

    # Run the agent
    result = await Runner.run(support_agent, messages)
    agent_response = result.final_output

    # Check for transfer signal
    if "TRANSFER_SIGNAL::" in agent_response:
        parts = agent_response.split("::")
        transfer_number = parts[1]
        response = VoiceResponse()
        response.say("Let me transfer you now. Please hold.")
        response.dial(transfer_number)
        return Response(content=str(response), media_type="application/xml")

    # Normal response
    history.append({"role": "assistant", "content": agent_response})
    sessions[caller] = history

    response = VoiceResponse()
    response.say(agent_response)
    gather = Gather(
        input="speech",
        action="/voice/process",
        speech_timeout="auto",
    )
    response.append(gather)
    return Response(content=str(response), media_type="application/xml")

if __name__ == "__main__":
    import uvicorn
    uvicorn.run("app.main:app", host="0.0.0.0", port=8000, reload=True)

Step 6: Configure Twilio and Test

Start your server and expose it with ngrok:

# Terminal 1
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

# Terminal 2
ngrok http 8000

In the Twilio console, set your phone number's Voice webhook to https://your-ngrok-url.ngrok.io/voice/incoming with HTTP POST.

Call your Twilio number and test these scenarios:

1. Order inquiry — Ask about order status by number
2. Policy question — Ask about the return policy
3. Escalation — Request to speak to a manager
4. Ticket creation — Report a damaged item that needs follow-up

Production Hardening Checklist

Before deploying to production, address these critical items:

• Replace in-memory sessions with Redis or a database-backed session store
• Add authentication to the Twilio webhook using request signature validation
• Implement rate limiting to prevent abuse
• Add structured logging with correlation IDs for each call
• Set up monitoring for agent latency, tool call failures, and transfer rates
• Add a fallback if the OpenAI API is unreachable — transfer to a human queue immediately
• Use connection pooling for PostgreSQL instead of creating new connections per request

FAQ

How do I handle multiple concurrent calls?

FastAPI is async by default, and the OpenAI Agents SDK supports async execution through Runner.run(). Each call gets its own session in the sessions dictionary. For production, replace the in-memory store with Redis to support horizontal scaling across multiple server instances.

Can I add more tools without changing the agent?

Yes. The Agents SDK dynamically adapts to whatever tools you provide. Simply create a new function with the @function_tool decorator and add it to the tools list in the agent definition. The agent will automatically discover when to use the new tool based on its docstring.

What happens if a tool call fails?

The Agents SDK includes built-in error handling. If a tool raises an exception, the error message is passed back to the agent, which can then decide how to proceed — usually by apologizing to the customer and either retrying or escalating. You should add try/except blocks in your tools and return user-friendly error messages.

How much does this cost to run per call?

At current OpenAI pricing, a typical 5-minute support call with 3-4 tool calls costs approximately $0.05-0.15 in API fees. Twilio voice costs about $0.013 per minute. The total per-call cost of $0.10-0.25 is significantly cheaper than the $5-15 cost of a human agent handling the same call.