How to Build a HIPAA-Grade Voice Receptionist (Encryption + Audit Logs)

TL;DR — HIPAA compliance for a voice agent is a system property, not a flag. You need a BAA with every PHI processor (OpenAI Enterprise, Twilio Enterprise, your cloud), AES-256 at rest, TLS 1.3 in transit, immutable audit logs, and PHI redaction before any non-BAA service sees the data.

What you'll build

A voice receptionist that takes patient calls, looks up appointments by name + DOB, transfers to a human when needed, and writes a tamper-evident audit trail of every PHI access. By the end you'll have a Twilio + OpenAI Realtime stack with the controls a HIPAA auditor expects to see.

Prerequisites

1. Twilio Enterprise account with signed BAA (standard tier ineligible).
2. OpenAI Enterprise tier with BAA (Realtime included).
3. AWS / GCP account with BAA — RDS Postgres, S3 with object lock, KMS.
4. Node 20+ or Python 3.11+ with TLS 1.3.
5. A compliance officer signing off on your data flow diagram.

Architecture

flowchart TD
  PSTN[Patient Call] --> TW[Twilio Enterprise BAA]
  TW -- TLS 1.3 --> APP[App Pod k3s]
  APP -- TLS 1.3 --> OAI[OpenAI Enterprise BAA]
  APP --> DB[(RDS Postgres KMS)]
  APP --> S3[(S3 ObjectLock)]
  APP --> AUDIT[(Audit Log append-only)]
  AUDIT --> SIEM[SIEM 6yr retention]

Step 1 — BAA matrix (do this before code)

Service	BAA Required?	Notes
Twilio Voice	Yes (Enterprise)	Standard tier is NOT BAA-eligible
OpenAI Realtime	Yes (Enterprise)	Verify in DPA
AWS RDS / S3 / KMS	Yes (covered by AWS BAA)	Enable encryption at rest
Logging / SIEM	Yes if PHI flows in	Datadog has BAA tier
Slack / email alerts	NEVER for PHI	Send only call IDs

Step 2 — Encryption at rest with KMS-wrapped keys

```ts // db.ts — Prisma + AWS KMS envelope encryption import { KMSClient, EncryptCommand, DecryptCommand } from "@aws-sdk/client-kms"; const kms = new KMSClient({ region: "us-east-1" });

export async function encryptPHI(plaintext: string): Promise { const { CiphertextBlob } = await kms.send(new EncryptCommand({ KeyId: process.env.KMS_KEY_ID!, Plaintext: Buffer.from(plaintext), })); return Buffer.from(CiphertextBlob!).toString("base64"); }

export async function decryptPHI(ciphertext: string): Promise { const { Plaintext } = await kms.send(new DecryptCommand({ CiphertextBlob: Buffer.from(ciphertext, "base64"), })); return Buffer.from(Plaintext!).toString("utf-8"); } ```

Apply per-field: encrypt patient name, DOB, MRN; leave call IDs in cleartext for queryability.

Step 3 — Append-only audit log

Every PHI read or write must be logged with: actor (agent ID), action, resource ID, timestamp, prior-row hash. Use a tamper-evident chain.

```sql CREATE TABLE audit_log ( id BIGSERIAL PRIMARY KEY, ts TIMESTAMPTZ NOT NULL DEFAULT now(), actor TEXT NOT NULL, action TEXT NOT NULL, resource TEXT NOT NULL, metadata JSONB, prev_hash BYTEA, hash BYTEA NOT NULL ); CREATE INDEX ON audit_log (resource, ts); ```

```ts import { createHash } from "crypto"; async function audit(actor: string, action: string, resource: string, meta: object) { const last = await db.audit_log.findFirst({ orderBy: { id: "desc" }}); const payload = JSON.stringify({ actor, action, resource, meta, ts: new Date().toISOString() }); const hash = createHash("sha256") .update((last?.hash ?? Buffer.alloc(0)) as any) .update(payload) .digest(); await db.audit_log.create({ data: { actor, action, resource, metadata: meta, prev_hash: last?.hash, hash }, }); } ```

Step 4 — PHI redaction before logs and traces

Before ANY trace, log line, or non-BAA service sees the transcript, redact PHI:

```ts const PHI_PATTERNS = [ { name: "ssn", re: /\b\d{3}-\d{2}-\d{4}\b/g }, { name: "dob", re: /\b(0[1-9]|1[0-2])/(0[1-9]|[12]\d|3[01])/\d{4}\b/g }, { name: "phone", re: /\b\d{3}[-.\s]?\d{3}[-.\s]?\d{4}\b/g }, ]; export function redact(text: string): string { let out = text; for (const p of PHI_PATTERNS) out = out.replace(p.re, `<${p.name}>`); return out; } ```

Step 5 — Identity verification before disclosure

Never disclose appointment details until the caller verifies name + DOB. Encode this in the system prompt and gate the tool:

```ts const lookupAppointment = tool({ name: "lookup_appointment", description: "Look up appointment ONLY after verifying name + DOB.", parameters: z.object({ name: z.string(), dob: z.string(), verified: z.boolean() }), execute: async ({ name, dob, verified }) => { if (!verified) return JSON.stringify({ error: "Verify identity first" }); await audit("agent", "read_phi", "appointment", { name_hash: hash(name) }); return JSON.stringify(await db.appointment.findFirst({ where: { name, dob } })); }, }); ```

Step 6 — Recording, retention, and right-to-delete

• Store call audio in S3 with Object Lock (compliance mode), 6-year retention.
• Index transcripts in Postgres with KMS-encrypted patient_id column.
• Build a /api/admin/phi/delete endpoint that nullifies PHI fields and logs the action; preserve the audit trail.

Common pitfalls

• Standard Twilio tier: not HIPAA-eligible. You MUST be on Enterprise with a signed BAA.
• Logging full transcripts to Datadog Standard: their BAA tier is separate. Redact or use the BAA-covered SKU.
• Storing recordings in default S3: enable Object Lock or it's not tamper-evident.
• Letting the model see SSN in the prompt history: redact before session.update instructions.

How CallSphere does this in production

CallSphere's Healthcare vertical (HIPAA + SOC 2) runs OpenAI Realtime PCM16 24kHz with server VAD on Twilio Enterprise + AWS BAA. Every PHI access writes to a tamper-evident chain audit log, recordings live in S3 Object Lock for 6 years, and the Salon vertical specifically does NOT touch this stack to keep blast radius small. See healthcare; we publish our SOC 2 report on request via /contact.

FAQ

Is OpenAI Realtime HIPAA-compliant by default? No — only with the Enterprise BAA. Standard API keys don't cover PHI.

Can I use ElevenLabs for healthcare? Yes if you're on the enterprise tier with a signed BAA — verify before sending PHI.

6-year retention or longer? HIPAA minimum is 6 years from creation date or last effective date. State laws can require more (NY: 6, TX: 10).

Audit log immutability — append-only is enough? Combined with hash-chaining and offsite backup yes. Add WORM storage (S3 Object Lock) for paranoid compliance.

Sources

• HHS HIPAA security rule
• Twilio HIPAA BAA
• OpenAI enterprise privacy
• AWS HIPAA whitepaper
• S3 Object Lock