Unit Testing AI Agents: Mocking LLM Calls for Fast, Deterministic Tests
Learn how to mock LLM API calls in your AI agent tests using FakeLLM objects, response fixtures, and assertion patterns for fast, deterministic, cost-free unit tests.
Agentic AI, LLM engineering, and the models behind modern automation — multi-agent systems, LLM evaluation and comparisons, RAG, fine-tuning, AI infrastructure, security, and production AI engineering.
From the blog
Learn how to mock LLM API calls in your AI agent tests using FakeLLM objects, response fixtures, and assertion patterns for fast, deterministic, cost-free unit tests.
Learn how to manage framework upgrades for AI agent systems. Covers semantic versioning, compatibility testing, shim layers for breaking changes, and gradual adoption strategies.
Learn how to safely upgrade LLM models in production systems. Covers evaluation frameworks, prompt adaptation, cost impact analysis, and progressive rollout strategies.
Implement accurate usage-based billing for an AI agent SaaS platform, including real-time metering of LLM tokens and API calls, Stripe integration for invoicing, and strategies for cost transparency.
Learn how to define user cohorts for AI agent interactions, perform retention analysis, cluster users by behavior patterns, and use cohort insights to personalize agent responses and improve outcomes.
Design a user onboarding flow that takes customers from sign-up to a working AI agent in under five minutes, including template selection, guided prompt configuration, and first-conversation testing.
Build AI agents that learn and adapt to individual user preferences over time — from implicit signal extraction and profile building to personalized responses — while respecting privacy boundaries.
Build comprehensive validation layers for LLM outputs using Pydantic validators, cross-field validation, domain-specific constraints, and data quality scoring. Catch hallucinations before they reach your database.
A practical comparison of four popular vector databases for RAG — pgvector, Pinecone, Chroma, and Weaviate — covering setup, indexing, query performance, and when to choose each one.