--- title: "Building a Clinical Documentation Agent: AI-Assisted Medical Note Generation" description: "Build an AI agent that generates structured clinical notes from encounter transcriptions, using SOAP format, template filling, and physician review workflows to improve documentation quality." canonical: https://callsphere.ai/blog/building-clinical-documentation-agent-ai-assisted-medical-notes category: "Learn Agentic AI" tags: ["Healthcare AI", "Clinical Documentation", "SOAP Notes", "Medical Transcription", "Python"] author: "CallSphere Team" published: 2026-03-17T00:00:00.000Z updated: 2026-05-06T15:17:12.613Z --- # Building a Clinical Documentation Agent: AI-Assisted Medical Note Generation > Build an AI agent that generates structured clinical notes from encounter transcriptions, using SOAP format, template filling, and physician review workflows to improve documentation quality. ## The Documentation Burden Physicians spend roughly two hours on documentation for every one hour of patient care. This documentation burden is a leading cause of clinician burnout. A clinical documentation agent listens to the encounter (via transcription), extracts structured medical information, generates a SOAP note draft, and presents it for physician review — cutting documentation time by 50 to 70 percent. The agent does not replace the physician's clinical judgment. It handles the mechanical work of structuring information, allowing the physician to focus on accuracy and completeness during review. ## The SOAP Note Structure SOAP (Subjective, Objective, Assessment, Plan) is the standard format for clinical documentation. Each section has distinct content requirements: ```mermaid flowchart LR CALLER(["Patient or Caregiver"]) subgraph TEL["Telephony"] SIP["Twilio SIP and PSTN"] end subgraph BRAIN["Healthcare AI Agent"] STT["Streaming STT Deepgram or Whisper"] NLU{"Intent and Entity Extraction"} TOOLS["Tool Calls"] TTS["Streaming TTS ElevenLabs or Rime"] end subgraph DATA["Live Data Plane"] CRM[("CRM and Notes")] CAL[("Calendar and Schedule")] KB[("Knowledge Base and Policies")] end subgraph OUT["Outcomes"] O1(["Appointment booked"]) O2(["Prescription refill request"]) O3(["Triage to clinician"]) end CALLER --> SIP --> STT --> NLU NLU -->|Lookup| TOOLS TOOLS CRM TOOLS CAL TOOLS KB NLU --> TTS --> SIP --> CALLER NLU -->|Resolved| O1 NLU -->|Schedule| O2 NLU -->|Escalate| O3 style CALLER fill:#f1f5f9,stroke:#64748b,color:#0f172a style NLU fill:#4f46e5,stroke:#4338ca,color:#fff style O1 fill:#059669,stroke:#047857,color:#fff style O2 fill:#0ea5e9,stroke:#0369a1,color:#fff style O3 fill:#f59e0b,stroke:#d97706,color:#1f2937 ``` ```python from dataclasses import dataclass, field from typing import Optional from datetime import datetime @dataclass class SOAPNote: patient_id: str encounter_date: datetime provider_id: str subjective: str = "" objective: str = "" assessment: str = "" plan: str = "" icd_codes: list[str] = field(default_factory=list) cpt_codes: list[str] = field(default_factory=list) status: str = "draft" # draft, pending_review, signed review_comments: Optional[str] = None def to_formatted_note(self) -> str: return ( f"ENCOUNTER NOTE - {self.encounter_date.strftime('%Y-%m-%d')} " f"{'=' * 50} " f"SUBJECTIVE: {self.subjective} " f"OBJECTIVE: {self.objective} " f"ASSESSMENT: {self.assessment} " f"PLAN: {self.plan} " f"ICD-10: {', '.join(self.icd_codes)} " f"CPT: {', '.join(self.cpt_codes)} " ) ``` ## Transcript Processing Pipeline The documentation agent takes raw encounter transcription and extracts structured information in stages: ```python from enum import Enum class SpeakerRole(Enum): PROVIDER = "provider" PATIENT = "patient" NURSE = "nurse" @dataclass class TranscriptSegment: speaker: SpeakerRole text: str timestamp: float class TranscriptProcessor: """Extracts structured clinical data from encounter transcripts.""" SYMPTOM_KEYWORDS = [ "pain", "ache", "fever", "cough", "nausea", "fatigue", "dizziness", "swelling", "rash", "bleeding", "shortness of breath", ] MEDICATION_PATTERNS = [ "taking", "prescribed", "started", "stopped", "increased", "decreased", ] def extract_chief_complaint(self, segments: list[TranscriptSegment]) -> str: for segment in segments: if segment.speaker == SpeakerRole.PROVIDER: if "what brings you in" in segment.text.lower() or "how can i help" in segment.text.lower(): idx = segments.index(segment) if idx + 1 list[dict]: symptoms = [] for segment in segments: if segment.speaker != SpeakerRole.PATIENT: continue text_lower = segment.text.lower() for keyword in self.SYMPTOM_KEYWORDS: if keyword in text_lower: symptoms.append({ "symptom": keyword, "context": segment.text, "timestamp": segment.timestamp, }) return symptoms def extract_medication_mentions(self, segments: list[TranscriptSegment]) -> list[dict]: mentions = [] for segment in segments: text_lower = segment.text.lower() for pattern in self.MEDICATION_PATTERNS: if pattern in text_lower: mentions.append({ "speaker": segment.speaker.value, "context": segment.text, "action": pattern, }) break return mentions ``` ## SOAP Note Generator The generator assembles extracted data into a structured note: ```python class SOAPNoteGenerator: def __init__(self, processor: TranscriptProcessor): self.processor = processor def generate( self, segments: list[TranscriptSegment], patient_id: str, provider_id: str, vitals: Optional[dict] = None, ) -> SOAPNote: chief_complaint = self.processor.extract_chief_complaint(segments) symptoms = self.processor.extract_symptoms(segments) medications = self.processor.extract_medication_mentions(segments) subjective = self._build_subjective(chief_complaint, symptoms, medications) objective = self._build_objective(vitals) return SOAPNote( patient_id=patient_id, encounter_date=datetime.utcnow(), provider_id=provider_id, subjective=subjective, objective=objective, assessment="[PENDING PROVIDER REVIEW]", plan="[PENDING PROVIDER REVIEW]", status="draft", ) def _build_subjective( self, chief_complaint: str, symptoms: list[dict], medications: list[dict] ) -> str: lines = [f"Chief Complaint: {chief_complaint}"] if symptoms: symptom_list = list({s['symptom'] for s in symptoms}) lines.append(f"Associated Symptoms: {', '.join(symptom_list)}") if medications: lines.append("Medication Discussion:") for med in medications: lines.append(f" - {med['context'][:100]}") return " ".join(lines) def _build_objective(self, vitals: Optional[dict]) -> str: if not vitals: return "[Vitals not yet recorded]" parts = [] if "bp" in vitals: parts.append(f"BP: {vitals['bp']}") if "hr" in vitals: parts.append(f"HR: {vitals['hr']}") if "temp" in vitals: parts.append(f"Temp: {vitals['temp']}") if "spo2" in vitals: parts.append(f"SpO2: {vitals['spo2']}") if "weight" in vitals: parts.append(f"Weight: {vitals['weight']}") return "Vitals: " + ", ".join(parts) ``` ## Review Workflow The generated note is always a draft. The physician must review and sign: ```python @dataclass class ReviewAction: action: str # "approve", "edit", "reject" provider_id: str timestamp: datetime edits: Optional[dict] = None comments: Optional[str] = None class NoteReviewWorkflow: def __init__(self): self.audit_trail: list[ReviewAction] = [] def submit_for_review(self, note: SOAPNote) -> SOAPNote: note.status = "pending_review" return note def process_review(self, note: SOAPNote, action: ReviewAction) -> SOAPNote: self.audit_trail.append(action) if action.action == "approve": note.status = "signed" elif action.action == "edit": if action.edits: for field_name, new_value in action.edits.items(): if hasattr(note, field_name): setattr(note, field_name, new_value) note.status = "pending_review" elif action.action == "reject": note.status = "draft" note.review_comments = action.comments return note ``` ## FAQ ### Can the documentation agent auto-populate the Assessment and Plan sections? The agent can suggest assessment and plan content based on the symptoms, history, and provider's documented treatment patterns. However, these sections require the most clinical judgment and should always be clearly marked as AI-suggested drafts. Many practices configure the agent to leave these sections blank for the physician to complete, generating only the Subjective and Objective sections automatically. ### How does the agent handle multiple conditions discussed in one visit? The agent identifies distinct clinical topics in the transcript and structures them as separate problem entries within the SOAP note. For example, if a patient discusses both knee pain and a medication refill for hypertension, the note will contain organized sections for each condition with their respective symptoms, findings, and plan items. ### What happens if the transcription quality is poor? The agent includes a confidence score for each extracted data point. Low-confidence extractions are flagged with brackets like "[unclear: possible mention of metformin]" so the reviewing physician knows to verify against their recollection. The agent never guesses — it surfaces uncertainty explicitly. --- #HealthcareAI #ClinicalDocumentation #SOAPNotes #MedicalTranscription #Python #AgenticAI #LearnAI #AIEngineering --- Source: https://callsphere.ai/blog/building-clinical-documentation-agent-ai-assisted-medical-notes