Android Developers: Insights into agent code generation from Android Bench

AI models can generate entire features in minutes, but is the code correct? We built Android Bench, our AI evaluation benchmark, to answer this question after analyzing thousands of generated code changes.



This talk shares insights from our journey, including all the ways we saw models cheats, how prompt specificity leads to higher code quality and how different agents approach the same task.



Resources:

Android Bench 2.0 → https://goo.gle/3TMN59O



Speakers: Tomáš Mlynarič



Chapters:

0:00 - Introduction to Android Bench

2:40 - Introducing Android Bench 2.0

5:05 - Grading challenge

9:34 - Cheating challenge

12:29 - Insights from benchmark results

18:27 - How models test their own work

21:42 - Pass rates versus completion rates

23:48 - Benchmark comparison and wrap-up



Watch the tools & performance sessions from Android Dev Summit 2026 → https://goo.gle/ADS26-Tools-and-Perf

Watch all the sessions from Android Dev Summit 2026 → https://goo.gle/ADS26-All-Sessions





Subscribe to Android Developers → https://goo.gle/AndroidDevs





#AndroidDevSummit2026





Event: Android Dev Summit 2026



Products Mentioned: Android Bench, Developer Tools