Natalie Glance on AI budget
From KDD 2025 - Applied Data Science Invited Speakers: Natalie Glance · · Association for Computing Machinery (ACM)
“We gave this team working on video call a budget of millions of dollars. You can spend up to this amount and you can do it by either spending a lot of time on cost optimization early in the year so you can save up money for fancier models down the line, but you figure it out. They're not super happy about this.”
On , Natalie Glance, Chief Engineering Officer at DUOLINGO INC, spoke about AI budget during KDD 2025 - Applied Data Science Invited Speakers: Natalie Glance on Association for Computing Machinery (ACM).
About Natalie Glance
Natalie Glance, Chief Engineering Officer at Duolingo, has spoken about the company's use of generative AI to improve learning efficacy. At KDD 2025, she described a fine-tuning approach for language models that involved sampling overly difficult conversations, simplifying the responses, and tuning the model, which she said made about 70% of sessions easier for learners. She also discussed the "video call with Lily" feature, which uses AI to conduct natural conversations with learners, and noted that the team focuses on "total words spoken" as a proxy for engagement. Glance stated that the company gave the video call team a budget of millions of dollars for AI spending, allowing them to decide how to allocate funds between cost optimization and model quality. At Config 2025, Glance discussed Duolingo's engineering and design culture, noting that the company outlawed the term "MVP" in favor of "V1" to emphasize quality while pragmatically cutting corners. She said the company runs hundreds of A/B tests in parallel and described a recent change replacing the "hearts" mechanic with an "energy" system, which she said was initially met with little leadership support but proved positive for both monetization and daily active user growth. Glance also mentioned that designers are starting to use AI for "vibe coding" complex animations to better communicate their vision to engineers.