What’s happened? Google is reportedly working on a new Gemini tool (reportedly called “Circle Screen”) that can analyze your phone’s screen and automatically explain or summarize it.
- The tool may effectively replace one of the most popular and celebrated AI-based features on smartphones: Circle to Search (via Android Authority).
- Google’s new screen-understanding tool lets you ask Gemini about whatever appears on your phone’s display, including text, images, and mathematical formulas, among other things.
Why is this important? Currently, users have to invoke the Circle to Search tool, highlight a portion of the screen, then receive relevant responses and the opportunity to ask follow-up queries.
- However, the new update builds on the gesture-based interface and extends it.
- Instead of launching a search, Gemini now interprets the selection and produces AI-based summaries or explanations.
- It marks a shift from simple visual search to an AI-powered experience in which your phone provides more details about what you see without invoking Circle to Search and switching interfaces.
Why should I care? The addition comes as part of Google’s broader effort to integrate Gemini across most touchpoints in Android (and even Wear OS).
- The new Gemini tool could streamline how we consume information on smartphones, eliminating the need to copy/paste, switching apps, or using the Circle to Search tool.
- For example, imagine reading a social-media post in another language and instantly getting a translation or explanation on the screen with a gesture, without leaving the app.
- Students or researchers could take a screenshot of a diagram, quote, or paragraph and ask Gemini to summarize or explain it.
Android Authority 2 / YouTube
OK, what’s next? We should expect more Android phones, not just the high-end flagships or Pixel devices, to get this Gemini-powered screen-analysis feature.
- Over time, this could change how we use Gemini, as they could get an AI explanation with a single gesture.
- However, the feature also raises questions about privacy and UX control as devices become better at understanding what’s on your screen.
- Most recently, Google was spotted integrating AI Mode to Circle to Search for providing answers to follow-up queries.