Evals Driven-Development: Engineering a Mental Health AI Coach Ethically & Safely
In the world of AI Mental Health, vibes can be dangerous with real consequences. Building Sondermind’s Mental Health AI Coach required us to invent a new playbook for Eval-Driven Development in order to balance effectiveness and safety. This session is for the builders who want to see how to handle the most difficult edge cases in the agentic world. We’ll show how we’ve built a Clinical Feedback Loop that turns human therapist insights into machine-readable evaluations in a production system with thousands of conversations. We’ll dive into: - The Ethics Engine: Building and calibrating modular guardrails that can be updated as clinical guidelines evolve. - Agentic Oversight: Why we moved from single-prompt agents to a closed-loop Supervisor/Executor/Evaluator pattern to ensure clinical adherence. - Human Oversight: How we monitor Sonder to ensure that we can improve safety and quality with clinical feedback. Speakers: Akele Reed — Sondermind; Dave Revere — SonderMind; Doug Keller — SonderMind.
About the Evals Track
Evals sessions at AI Engineer World's Fair 2026 in San Francisco.
When
Wednesday, July 1, 2026
2:25 PM - 2:45 PM·20m
Where
Track 5 · Room 2005
Capacity: 250 attendees
Sign in to add this talk to your schedule.
Speakers (3)
Akele Reed
Sondermind
Principal AI Engineer at SonderMind, Akele Reed leads the team behind the company's conversational AI mental health feature and has served as a primary architect of its guardrails and evaluations framework. Her work sits at the intersection of AI capability and responsibility, designing systems that earn trust through rigorous safety pipelines, human expert feedback loops, and continuous oversight in one of the highest-stakes domains in healthcare. Akele is passionate about making AI trustworthy not just in theory, but in production and building the infrastructure and culture that allow engineers and clinicians alike to confidently rely on AI-powered tools. She brings over nine years of experience in applied AI and model training, including her previous role at 23andMe, and holds a Master's degree in Computer Science from Georgia Tech. Away from the screens, she enjoys hiking, beekeeping, and baking.
