Insights into agent code generation from Android Bench

AI models can generate entire features in minutes, but is the code correct? We built Android Bench, our AI evaluation benchmark, to answer this question after analyzing thousands of generated code changes. This talk shares insights from our journey, including all the ways we saw models cheats, how prompt specificity leads to higher code quality and how different agents approach the same task. Resources: Android Bench 2.0 → https://goo.gle/3TMN59O Speakers: Tomáš Mlynarič Chapters: 0:00 - Introduction to Android Bench 2:40 - Introducing Android Bench 2.0 5:05 - Grading challenge 9:34 - Cheating challenge 12:29 - Insights from benchmark results 18:27 - How models test their own work 21:42 - Pass rates versus completion rates 23:48 - Benchmark comparison and wrap-up Watch the tools & performance sessions from Android Dev Summit 2026 → https://goo.gle/ADS26-Tools-and-Perf Watch all the sessions from Android Dev Summit 2026 → https://goo.gle/ADS26-All-Sessions Subscribe to Android Developers → https://goo.gle/AndroidDevs #AndroidDevSummit2026 Event: Android Dev Summit 2026 Products Mentioned: Android Bench, Developer Tools