Contextual Follow-Up Questions: Building Agents That Ask Smart Clarifying Questions

The Art of Asking the Right Question

The difference between a helpful assistant and an annoying one often comes down to questions. A great agent asks precisely the right question at the right time — one that fills a genuine information gap and moves the conversation forward. A poor agent asks too many questions, asks obvious ones, or asks things the user already answered.

Contextual follow-up questions are dynamically generated based on what the agent already knows, what it still needs, and the specific task being performed.

Modeling Information Gaps

Start by defining what information is needed for each task and tracking what has been gathered so far.

flowchart LR
    INPUT(["User intent"])
    PARSE["Parse plus<br/>classify"]
    PLAN["Plan and tool<br/>selection"]
    AGENT["Agent loop<br/>LLM plus tools"]
    GUARD{"Guardrails<br/>and policy"}
    EXEC["Execute and<br/>verify result"]
    OBS[("Trace and metrics")]
    OUT(["Outcome plus<br/>next action"])
    INPUT --> PARSE --> PLAN --> AGENT --> GUARD
    GUARD -->|Pass| EXEC --> OUT
    GUARD -->|Fail| AGENT
    AGENT --> OBS
    style AGENT fill:#4f46e5,stroke:#4338ca,color:#fff
    style GUARD fill:#f59e0b,stroke:#d97706,color:#1f2937
    style OBS fill:#ede9fe,stroke:#7c3aed,color:#1e1b4b
    style OUT fill:#059669,stroke:#047857,color:#fff

from dataclasses import dataclass, field
from typing import Optional
from enum import Enum

class GapPriority(Enum):
    BLOCKING = "blocking"    # Cannot proceed without this
    IMPORTANT = "important"  # Significantly improves outcome
    OPTIONAL = "optional"    # Nice to have

@dataclass
class InformationGap:
    field_name: str
    description: str
    priority: GapPriority
    question_template: str
    context_hints: list[str] = field(default_factory=list)
    max_asks: int = 2
    times_asked: int = 0

    def can_ask(self) -> bool:
        return self.times_asked < self.max_asks

@dataclass
class TaskRequirements:
    task_name: str
    gaps: list[InformationGap]
    known_info: dict = field(default_factory=dict)

    def blocking_gaps(self) -> list[InformationGap]:
        return [
            g for g in self.gaps
            if g.field_name not in self.known_info
            and g.priority == GapPriority.BLOCKING
            and g.can_ask()
        ]

    def important_gaps(self) -> list[InformationGap]:
        return [
            g for g in self.gaps
            if g.field_name not in self.known_info
            and g.priority == GapPriority.IMPORTANT
            and g.can_ask()
        ]

    def completion_ratio(self) -> float:
        total = len(self.gaps)
        filled = sum(
            1 for g in self.gaps if g.field_name in self.known_info
        )
        return filled / total if total > 0 else 1.0

Smart Question Generation

Questions should incorporate context from what the agent already knows to demonstrate it has been listening and to avoid redundant requests.

class QuestionGenerator:
    def __init__(self):
        self.conversation_context: dict = {}

    def generate(
        self, gap: InformationGap, known_info: dict
    ) -> str:
        question = gap.question_template

        # Inject known context into the question
        for key, value in known_info.items():
            placeholder = "{" + key + "}"
            if placeholder in question:
                question = question.replace(placeholder, str(value))

        # Add contextual hints based on known information
        hints = self._select_hints(gap, known_info)
        if hints:
            question += f" ({hints})"

        gap.times_asked += 1
        return question

    def _select_hints(
        self, gap: InformationGap, known_info: dict
    ) -> Optional[str]:
        relevant_hints = []
        for hint in gap.context_hints:
            # Hints reference known info keys
            for key in known_info:
                if key in hint:
                    filled = hint.replace(
                        f"{{{key}}}", str(known_info[key])
                    )
                    relevant_hints.append(filled)
        return "; ".join(relevant_hints) if relevant_hints else None

The Clarification Controller

The controller decides when to ask, what to ask, and when to stop asking and proceed with available information.

class ClarificationController:
    def __init__(
        self,
        max_questions_per_turn: int = 1,
        proceed_threshold: float = 0.7,
    ):
        self.generator = QuestionGenerator()
        self.max_per_turn = max_questions_per_turn
        self.proceed_threshold = proceed_threshold
        self.questions_this_session = 0
        self.max_session_questions = 5

    def should_ask(self, requirements: TaskRequirements) -> bool:
        if self.questions_this_session >= self.max_session_questions:
            return False

        # Always ask if blocking gaps exist
        if requirements.blocking_gaps():
            return True

        # Ask important gaps only if below threshold
        if requirements.completion_ratio() < self.proceed_threshold:
            return bool(requirements.important_gaps())

        return False

    def get_questions(
        self, requirements: TaskRequirements
    ) -> list[str]:
        questions = []

        # Blocking gaps first
        for gap in requirements.blocking_gaps():
            if len(questions) >= self.max_per_turn:
                break
            q = self.generator.generate(gap, requirements.known_info)
            questions.append(q)
            self.questions_this_session += 1

        # Then important gaps if room
        if len(questions) < self.max_per_turn:
            for gap in requirements.important_gaps():
                if len(questions) >= self.max_per_turn:
                    break
                q = self.generator.generate(
                    gap, requirements.known_info
                )
                questions.append(q)
                self.questions_this_session += 1

        return questions

Example: Travel Booking Agent

travel_task = TaskRequirements(
    task_name="book_flight",
    gaps=[
        InformationGap(
            "destination", "Where the user wants to fly",
            GapPriority.BLOCKING,
            "Where would you like to fly to?",
        ),
        InformationGap(
            "departure_date", "When to depart",
            GapPriority.BLOCKING,
            "When would you like to depart for {destination}?",
            context_hints=["popular travel period for {destination}"],
        ),
        InformationGap(
            "return_date", "When to return",
            GapPriority.IMPORTANT,
            "When would you like to return from {destination}?",
        ),
        InformationGap(
            "cabin_class", "Preferred cabin class",
            GapPriority.OPTIONAL,
            "Any preference on cabin class?",
        ),
    ],
)

controller = ClarificationController(max_questions_per_turn=1)

# User says: "I want to fly to Tokyo"
travel_task.known_info["destination"] = "Tokyo"

if controller.should_ask(travel_task):
    questions = controller.get_questions(travel_task)
    for q in questions:
        print(q)
    # Output: "When would you like to depart for Tokyo?"

The question naturally incorporates the already-known destination, making it feel like a real conversation rather than an interrogation.

FAQ

How many clarifying questions should an agent ask before proceeding?

Limit clarifying questions to one per turn and five per session. After that, proceed with defaults or partial information and let the user refine. Research shows that more than two consecutive questions causes significant user drop-off, so interleave questions with partial answers when possible.

How do you handle users who refuse to answer a clarifying question?

If a user ignores a blocking question, rephrase it once with different wording. If they ignore it again, explain why the information is needed and offer alternatives. For example: "I need the date to search flights. Would you like me to show options for the next week instead?" Providing a default path prevents dead ends.

Should agents ask optional questions at all?

Ask optional questions only when the conversation is flowing well and the user seems engaged. If the user is giving terse responses or showing impatience signals, skip optional gaps and use sensible defaults. The agent should track engagement signals like response length and response time to calibrate.

---

#FollowUpQuestions #Clarification #DialogFlow #ConversationalAI #Python #AgenticAI #LearnAI #AIEngineering