Agentic AI, LLM engineering, and the models behind modern automation — multi-agent systems, LLM evaluation and comparisons, RAG, fine-tuning, AI infrastructure, security, and production AI engineering.
From the blog
Master error handling in async Python for AI agents. Learn asyncio.timeout, task cancellation, cleanup patterns, and exception groups for robust agent systems.
Design error messages for AI agents that categorize failures, provide helpful recovery paths, maintain user trust during outages, and turn mistakes into positive experiences.
Build AI agents that detect their own errors, apply correction strategies, and learn from failures through feedback loops. Covers error detection, self-correction, escalation paths, and continuous improvement.
Implement comprehensive error tracking for AI agent systems with error classification, severity-based alert routing to Sentry and PagerDuty, and incident response workflows tailored to LLM failure modes.
Build an ETL pipeline that extracts conversation logs from AI agent systems, anonymizes PII, transforms them into training-ready formats, and filters for quality to improve agent performance.
Learn a comprehensive evaluation methodology for fine-tuned LLMs, combining automated benchmarks, human evaluation, and production A/B testing to measure real-world improvement with statistical rigor.
Learn how to build comprehensive RAG evaluation systems with online metrics, user feedback loops, automated quality scoring, A/B testing, and degradation detection for production retrieval pipelines.
Learn how to design, label, and maintain evaluation datasets for AI agents, covering dataset structure, diversity requirements, edge cases, and ongoing maintenance strategies.
Learn how to build evaluation frameworks with scoring rubrics, A/B testing, and regression testing to systematically improve prompt quality and catch regressions before production.
