Building a Size and Fit Agent: AI-Powered Sizing Recommendations for Fashion Retail

The Sizing Problem in Online Fashion

Size-related returns account for 30 to 40 percent of all fashion e-commerce returns. A "Medium" from one brand fits like a "Large" from another. Customers cannot try items on, so they either order multiple sizes or guess — both outcomes are expensive for retailers. An AI sizing agent solves this by mapping a customer's measurements and preferences to brand-specific sizing data.

Modeling Size Charts

The foundation is a structured representation of brand sizing data. Each brand-product combination maps size labels to measurement ranges in centimeters.

flowchart LR
    REL(["Release of<br/>Building a Size and Fit<br/>Agent"])
    NEW1["What's new<br/>flagship feature 1"]
    NEW2["What's new<br/>flagship feature 2"]
    NEW3["What's new<br/>flagship feature 3"]
    BREAK{"Breaking<br/>changes?"}
    MIG["Migration steps"]
    UPG(["Upgrade now"])
    WAIT(["Pin current,<br/>upgrade later"])
    REL --> NEW1
    REL --> NEW2
    REL --> NEW3
    NEW1 --> BREAK
    NEW2 --> BREAK
    NEW3 --> BREAK
    BREAK -->|Yes| MIG --> UPG
    BREAK -->|No| UPG
    BREAK -->|Risk averse| WAIT
    style REL fill:#4f46e5,stroke:#4338ca,color:#fff
    style BREAK fill:#f59e0b,stroke:#d97706,color:#1f2937
    style UPG fill:#059669,stroke:#047857,color:#fff
    style WAIT fill:#0ea5e9,stroke:#0369a1,color:#fff

from agents import Agent, Runner, function_tool
from typing import Optional

# Brand sizing data: size -> measurement ranges in cm
SIZE_CHARTS = {
    "BrandA_T-Shirt": {
        "S":  {"chest": (86, 91),  "waist": (71, 76),  "length": 68},
        "M":  {"chest": (91, 97),  "waist": (76, 81),  "length": 71},
        "L":  {"chest": (97, 102), "waist": (81, 86),  "length": 74},
        "XL": {"chest": (102, 107), "waist": (86, 91), "length": 76},
    },
    "BrandB_T-Shirt": {
        "S":  {"chest": (88, 94),  "waist": (73, 78),  "length": 70},
        "M":  {"chest": (94, 100), "waist": (78, 84),  "length": 73},
        "L":  {"chest": (100, 106), "waist": (84, 90), "length": 76},
        "XL": {"chest": (106, 112), "waist": (90, 96), "length": 79},
    },
}

FIT_PREFERENCES = {
    "slim": -2,    # Subtract 2cm from measurements for tighter fit
    "regular": 0,
    "relaxed": 3,  # Add 3cm for looser fit
}

@function_tool
def recommend_size(brand_product: str, chest_cm: float,
                   waist_cm: float, fit_preference: str = "regular") -> str:
    """Recommend a size based on body measurements and fit preference."""
    chart = SIZE_CHARTS.get(brand_product)
    if not chart:
        return f"No sizing data available for {brand_product}."

    adjustment = FIT_PREFERENCES.get(fit_preference, 0)
    adjusted_chest = chest_cm - adjustment
    adjusted_waist = waist_cm - adjustment

    best_size = None
    best_score = float("inf")
    for size_label, measurements in chart.items():
        chest_range = measurements["chest"]
        waist_range = measurements["waist"]
        chest_mid = (chest_range[0] + chest_range[1]) / 2
        waist_mid = (waist_range[0] + waist_range[1]) / 2
        score = abs(adjusted_chest - chest_mid) + abs(adjusted_waist - waist_mid)
        if score < best_score:
            best_score = score
            best_size = size_label

    return (
        f"Recommended size for {brand_product}: {best_size} "
        f"(fit: {fit_preference}). Based on chest {chest_cm}cm, "
        f"waist {waist_cm}cm with {fit_preference} fit adjustment."
    )

Cross-Brand Size Mapping

Customers often know their size in one brand but not another. A mapping tool translates between brand size systems.

@function_tool
def map_size_across_brands(source_brand: str, source_size: str,
                           target_brand: str) -> str:
    """Map a known size from one brand to the equivalent in another."""
    source_chart = SIZE_CHARTS.get(source_brand)
    target_chart = SIZE_CHARTS.get(target_brand)
    if not source_chart or not target_chart:
        return "Sizing data not available for one or both brands."

    source_measurements = source_chart.get(source_size)
    if not source_measurements:
        return f"Size {source_size} not found for {source_brand}."

    # Find closest match in target brand
    source_chest_mid = sum(source_measurements["chest"]) / 2
    source_waist_mid = sum(source_measurements["waist"]) / 2

    best_size = None
    best_score = float("inf")
    for size_label, measurements in target_chart.items():
        chest_mid = sum(measurements["chest"]) / 2
        waist_mid = sum(measurements["waist"]) / 2
        score = abs(source_chest_mid - chest_mid) + abs(source_waist_mid - waist_mid)
        if score < best_score:
            best_score = score
            best_size = size_label

    return (
        f"Your {source_size} in {source_brand} maps to "
        f"{best_size} in {target_brand}."
    )

@function_tool
def get_fit_feedback_summary(product_id: str) -> str:
    """Get aggregated fit feedback from other customers."""
    # In production, query your reviews database
    feedback = {
        "total_reviews": 234,
        "runs_small_pct": 15,
        "true_to_size_pct": 72,
        "runs_large_pct": 13,
        "common_note": "Sleeves run slightly long",
    }
    return (
        f"Fit feedback for {product_id}: {feedback['true_to_size_pct']}% "
        f"say true to size, {feedback['runs_small_pct']}% runs small, "
        f"{feedback['runs_large_pct']}% runs large. "
        f"Note: {feedback['common_note']}"
    )

Assembling the Size Agent

size_agent = Agent(
    name="Size and Fit Advisor",
    instructions="""You are a sizing expert for an online fashion store.
    Help customers find their perfect size.

    Process:
    1. Ask for the customer's key measurements (chest, waist) in cm or inches
    2. Ask about their fit preference (slim, regular, relaxed)
    3. If they know their size in another brand, use cross-brand mapping
    4. Check fit feedback from other customers for the specific item
    5. Recommend a size with confidence level and explanation
    6. Mention the return policy for size exchanges

    Always convert inches to cm internally (1 inch = 2.54 cm).
    If uncertain between two sizes, recommend the larger one and
    explain why.""",
    tools=[recommend_size, map_size_across_brands, get_fit_feedback_summary],
)

result = Runner.run_sync(
    size_agent,
    "I wear a Medium in BrandA t-shirts. What size should I get in BrandB?",
)
print(result.final_output)

Reducing Returns with Confidence Scores

Add a confidence indicator that tells customers how reliable the recommendation is. High confidence means measurements fall squarely within a size range. Low confidence means the customer is between sizes and should consider their fit preference carefully.

def calculate_fit_confidence(chest_cm: float, waist_cm: float,
                             size_range: dict) -> float:
    """Return 0-100 confidence score for a size recommendation."""
    chest_low, chest_high = size_range["chest"]
    waist_low, waist_high = size_range["waist"]

    chest_in_range = chest_low <= chest_cm <= chest_high
    waist_in_range = waist_low <= waist_cm <= waist_high

    if chest_in_range and waist_in_range:
        return 95.0
    elif chest_in_range or waist_in_range:
        return 70.0
    else:
        return 45.0

FAQ

How do I handle customers who only know their measurements in inches?

Build unit conversion directly into the agent's measurement collection flow. When a customer provides measurements, detect whether the values are likely inches (typically 30-50 for chest) or centimeters (typically 76-127 for chest). Confirm the unit with the customer and convert to your internal standard. Store both the original and converted values.

How accurate are AI size recommendations compared to physical try-ons?

With good sizing data and customer measurements, AI recommendations achieve 80 to 85 percent accuracy for basic garments like t-shirts and pants. Accuracy drops for items with complex fits like blazers or dresses. Incorporating community fit feedback ("runs small") and the customer's historical return data improves accuracy to 90 percent or higher over time.

Should I store customer measurements for future visits?

Yes, with explicit consent. Stored measurements allow instant recommendations on return visits without re-measuring. Implement this as an opt-in profile feature with clear data privacy disclosures. Let customers update measurements anytime and delete their data on request to comply with GDPR and CCPA requirements.

#SizeRecommendation #FashionTech #FitPrediction #RetailAI #ReturnReduction #AgenticAI #LearnAI #AIEngineering