Zach Bratun-Glennon

Many LLM deployment conversations focus on models, benchmarks, and prompting, but the hardest problems actually start after the model works. In this session, Senior Director of AI Software at NVIDIA and former CEO of CentML Gennady Pekhimenko and Gradient General Partner Zach Bratun-Glennon will explore the details and cutting edge of inference performance. They'll unpack what actually happens when you try to run large models in production, including lessons and patterns observed from real deployments, and what the next generation of compilers, frameworks and platform acceleration should look like to enable successful AI workloads.

As demand for large scale AI systems grows, the limiting factor is no longer model capability, but compute availability, efficiency, and system design. Lambda cofounder/CEO Stephen Balaban and Gradient General Partner Zach Bratun-Glennon will examine how modern workloads interact with real-world compute constraints and the software investment and developments taking place to get the most out of cutting edge GPUs. They’ll unpack where training and inference workloads diverge in their infrastructure requirements, the practical limits of GPU utilization and what drives underperformance, and what we can expect to see next in AI infrastructure as constraints continue to evolve. Speakers: Zach Bratun-Glennon — Gradient; Stephen Balaban — Lambda.