Google Preps 'Smart Screenshots' Feature To Let You Search With a Screenshot

According to Abner Li from 9to5Google, Google is working on a new "Smart Screenshots" feature that integrates Google Lens abilities into the Google app's screenshot function. From the report: The Google app has long had an "Edit & share screenshots" ability where captures made within Search would reveal cropping and annotation tools. Meanwhile, Assistant has long maintained a "What's on my screen" capability that analyzes what you're currently viewing for search suggestions. Google app 10.61 reveals work on "Smart Screenshots" that combine those two features. Like before, a toolbar -- which interestingly uses a four-color light bar -- appears after you take a screenshot. A small preview is shown at the left with a pencil button overlaid. You can open the system share sheet, but the Google app also suggests a frequently used app.

The most interesting addition is Lens. "Exploring with Lens" could be intended as a "Screen search" replacement given that Lens is increasingly taking over visual lookup throughout first-party apps, like Chrome. After taking a capture, Smart Screenshots have an easy way to invoke Lens for search, OCR, and finding visually "similar items." The existing editing tools (Annotating, Cropping, and Sharing) will remain and this new functionality appears to even use the same settings toggle to enable. It's unclear if this functionality once live will again be limited to screenshots taken within Search, or if it will expand to be systemwide and invokable anywhere. A notification from the Google app could appear after capturing a screenshot.