![\"Image](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/Image-of-the-Android-Bot-mascot-on-the-Rabbit-R1.jpg\"\/ "\\"Image")
<\/p>\n<\/p>\n
Ryan Haines \/ Android Authority<\/p>\n
For a brief period last year, it seemed that AI-powered gadgets like the Rabbit R1<\/a> were going to be the next big thing. People were fascinated by the idea of replacing their smartphones with tiny wearable boxes they could talk to, but unfortunately, these gadgets failed to live up to the hype. They failed for a variety of reasons \u2014 like price, redundancy, and utility \u2014 but the seeds they planted for a future of fully automated apps stuck around.<\/p>\n These seeds grew into the \u201cagentic AI\u201d trend we see today. Companies are racing to build AI products that can perform tasks on your behalf, like helping you code a new project, booking an appointment, or ordering items online. As one of the leaders in the AI race, Google is also working on its own AI agents, with Gemini in Chrome<\/a> being its most notable offering.<\/p>\n Gemini in Chrome, of course, can only perform actions within the browser and not in other applications. If you want to automate Android apps, your options are limited to third-party tools like Tasker<\/a>, which often have a steep learning curve. Even then, these tools must be meticulously configured to take specific actions in predetermined apps. Unlike newer AI agents, these existing automation tools can\u2019t perform generalized tasks from a single, natural language prompt.<\/p>\n That\u2019s why Project Astra, Google\u2019s experimental universal AI project, is so exciting. At Google I\/O, the company showed off a version of Astra that can control your Android phone<\/a>. In the demo, the assistant found a document online, scrolled through it to find specific information, and then searched YouTube for related videos \u2014 all completely hands-free. To accomplish this, Astra recorded the screen for analysis and then sent tap or swipe inputs to launch apps or scroll through pages.<\/p>\n Google\u2019s demo showed the immense potential for an AI agent that can perform tasks in Android apps, but it also revealed that the company still has a lot of work to do. For starters, the parts of the video featuring the AI agent were sped up 2x, suggesting it\u2019s quite slow. This wasn\u2019t an issue in the scenario concocted for the demo, where the user clearly had their hands full, but it will be a problem in the real world. A slow agent means your phone will be occupied while it works, and common interruptions like a notification, an incoming call, or an alarm could disrupt the process by interfering with its screen analysis or inputs.<\/p>\n The purpose of the I\/O demo was simply to show off Project Astra\u2019s capabilities rather than detail how an on-device AI agent would actually work. Google had to hack together a prototype that took advantage of existing Android APIs in unintended ways \u2014 the MediaProjection API for screen recording and the Accessibility API for screen input \u2014 which resulted in the issues mentioned above.<\/p>\n Over the past few months, however, Google has been working on a new, standardized framework for AI agents to control Android apps. This framework, called Computer Control, is designed to enable automated control of Android apps in the background, sidestepping those problems. Although Google probably won\u2019t announce Computer Control until next year\u2019s Android 17<\/a> release, we managed to uncover a lot of information about it by digging through Android code. Here\u2019s what we know so far.<\/p>\n You\u2019re reading the Authority Insights Newsletter, a weekly newsletter that reveals some new facet of Android that hasn\u2019t been reported on anywhere else. If you\u2019re looking for the latest scoops, the hottest leaks, and breaking news on Google\u2019s Android operating system and other mobile tech topics, then we\u2019ve got you covered.<\/p>\n Subscribe here<\/a> to get this post delivered to your email inbox every Saturday.<\/p>\n How Google\u2019s Computer Control feature enables automated control of Android apps<\/p>\n With the release of Android 13 in 2022, Google introduced a new system-level service called the Virtual Device Manager<\/a> (VDM). This service enables the creation of virtual displays that are separate from the primary, visible display the user sees. Apps can be launched on these virtual displays and then streamed to a remote device. In turn, that remote device can send back input events like clicks or keyboard presses for the system to inject into the app.<\/p>\n VDM already forms the backbone of the App Streaming feature in Chrome OS<\/a>, which lets you stream an app from your Android phone to your Chromebook. It also enables the Connected Camera feature<\/a> on Pixel devices, as Google quietly upgraded the service in last year\u2019s release to support virtual cameras. The service plays a key role in Google\u2019s long-term efforts to improve cross-device compatibility, and it seems the company now aims to leverage it to power its new Computer Control feature.<\/p>\n Google has added code for starting a \u201cComputer Control Session\u201d to facilitate the automated control of Android apps. Each session involves a single trusted virtual display that hosts the app being automated, as well as virtual input devices for sending touch and key events.<\/p>\n Client apps using the Computer Control framework must specify the virtual display\u2019s properties, including its name, height, width, and density. They must also specify whether the display should remain unlocked and interactive even when the host device is locked \u2014 a crucial feature for truly unattended control. (However, the device must first be unlocked for an automation session to be initiated.)<\/p>\n Furthermore, client apps must set an output surface for the virtual display\u2019s content. This allows them to access the raw display frames, which can then be streamed to a remote connected device for analysis.<\/p>\n Another crucial component of this framework is its ability to mirror the trusted virtual display onto a separate, interactive virtual display. This interactive display can have different dimensions than the original, and the system will automatically map input events between them. This architecture allows users to see and manually interact with the app being automated, crucially without interfering with the automation process itself.<\/p>\n This separation is key. If the original trusted virtual display were mirrored directly to a user \u2014 on a connected PC, for example \u2014 then resizing the window could break the automation, as changing display dimensions can trigger a configuration change that forces apps to restart. By creating a second, interactive display that mirrors the trusted one, users can remotely view and send inputs to the app without disrupting the process.<\/p>\n Given the sensitive nature of this framework, access is restricted to highly privileged, trusted applications. To use it, an app must first hold the new ACCESS_COMPUTER_CONTROL permission. This permission can only be held by apps signed with a digital certificate that has been explicitly allowlisted in the OS. Apps holding this permission must then ask the user to explicitly approve their use of the Computer Control feature; this approval can be granted for a single session or for all future sessions.<\/p>\n Mishaal Rahman \/ Android Authority<\/p>\n Once an app\u2019s request to start a Computer Control session has been granted, the framework can restrict it from launching or interacting with apps other than the one being automated. This protects from automation sessions being exploited to access other sensitive apps on the device.<\/p>\n In practice, this means only select apps trusted by Google or your device\u2019s manufacturer will have access to the Computer Control framework. Regular apps won\u2019t be able to discreetly launch and control other apps in the background without your knowledge.<\/p>\n Computer Control \u2014 A new level of automation for Android apps?<\/p>\n While we can conclude that the Computer Control framework is designed to let trusted clients analyze screen data and automate tasks on your behalf, there\u2019s still a lot we don\u2019t know. For example, it\u2019s unclear how exactly these clients will control apps.<\/p>\n Does the \u201ccomputer\u201d in \u201cComputer Control\u201d literally refer to a PC, suggesting Google plans to stream apps to a remote PC or server for automation? This approach would be similar to how the Rabbit R1 works<\/a>, though in that case, the apps ran entirely on Rabbit\u2019s servers. Or does \u201ccomputer\u201d mean a robot in a more general sense, where an on-device AI model analyzes the screen and performs actions locally? The former is more likely given the architecture of the Computer Control framework and where the code resides, but the latter is also certainly possible when using a multimodal model like Gemini Nano<\/a>. This approach would also be more private, but it would consequently strain the device\u2019s memory and battery more.<\/p>\n Whatever the case, we\u2019re excited to see Google building a proper framework for true agentic AI on Android. The Computer Control framework opens the door to fully automating your apps, which is a big deal as it can not only save time but also dramatically improve accessibility. Of course, AI agents won\u2019t always get things right. This is likely why Google included the ability to mirror the automation onto an interactive display, allowing users to supervise the process and make adjustments as needed.<\/p>\n Again, our understanding of this feature comes from code we examined in the latest Android build (i.e., Android 16 QPR2 Beta 2<\/a>). We may have missed some details, and it\u2019s unclear when Google plans to launch it. If we learn more about Computer Control, we\u2019ll be sure to let you know \u2014 so consider subscribing to the Authority Insights Newsletter<\/a> so you don\u2019t miss a beat.<\/p>\n Want more?<\/p>\n Authority Insights is more than a newsletter \u2014 it\u2019s the hub for all our best content. If you care about Android, you won\u2019t want to miss any of our other exclusive reports<\/a>.<\/p>\n Don\u2019t have time to read them all? Subscribe to our Authority Insights Podcast<\/a> to hear me and my co-host, C. Scott Brown, break down our top stories of the week.<\/p>\n This week\u2019s top Authority Insights\n<\/p>\n The Pixel Watch 4 will bring new watch faces\u2026and potentially hypertension alerts?!\n<\/p>\n <\/p>\n Here are the 8 new watch faces coming with the Pixel Watch 4<\/p>\n Related<\/p>\n Google is preparing its own Apple-style hypertension alert system for the Pixel Watch<\/p>\n Related<\/p>\n Nano Banana, Aerial view, and more new features coming to Google apps\n<\/p>\n <\/p>\n Google’s viral image generator is coming to AI Mode<\/p>\n Related<\/p>\n This upcoming Google Maps feature could make virtual exploration easier, and here’s a look<\/p>\n Related<\/p>\n Google Photos’ Collage tool is about to get way more flexible and easier to use<\/p>\n Related<\/p>\n Google Photos will soon let you choose how you want your photos animated<\/p>\n Related<\/p>\n Here’s the latest progress on how Google Messages @mentions are going to work<\/p>\n Related<\/p>\n Gemini\u2019s home screen could soon get a Discovery-style redesign<\/p>\n Related<\/p>\n This simple change could make Gemini Scheduled Actions a joy to use<\/p>\n Related<\/p>\n Better late than never\u2026\n<\/p>\n <\/p>\n Android 16 QPR2 may let you flip your Pixel’s navigation bar like on Samsung phones (Update: Demo)<\/p>\n Related<\/p>\n Spotify is finally fixing one of its most frustrating shuffle problems<\/p>\n Related<\/p>\n Last minute leaks\u2026\n<\/p>\n <\/p>\n Samsung’s One UI 8.5 update is an even bigger deal than you thought<\/p>\n Related<\/p>\n Exclusive first look: This is the canceled Google Pixel 4 5G<\/p>\n Related<\/p>\n Other top stories\n<\/p>\n What the heck is going on with Samsung wearables?!\n<\/p>\n <\/p>\n Galaxy Ring battery scare leaves user stranded and hospitalized<\/p>\n Related<\/p>\n Either Galaxy Watch users are suddenly sleeping better, or Samsung changed how sleep scores work<\/p>\n Related<\/p>\n Are you willing to give Google Home another chance?\n<\/p>\n <\/p>\n Gemini for Home is here to transform how you use Google Home<\/p>\n Related<\/p>\n Fool me twice: Smart home users say they’re done trusting Google<\/p>\n Related<\/p>\n I used the new Google Home Speaker. Here\u2019s why you should save your money<\/p>\n Related<\/p>\n Paying for Google Home Premium is an absolute non-starter for me<\/p>\n Related<\/p>\n The more we learn about Android\u2019s sideloading restrictions, the more worried we get\n<\/p>\n <\/p>\n Google’s new rules could wipe out sideloading and alternative app stores, F-Droid warns<\/p>\n Related<\/p>\n We finally know how Android’s new app verification rules will actually work<\/p>\n Related<\/p>\n Check out these new features!\n<\/p>\n <\/p>\n Your Pixel phones can now easily connect to two earbuds for simultaneous audio<\/p>\n Related<\/p>\n The Galaxy Tab S11’s best new feature turns it into a powerful Linux computer<\/p>\n Related<\/p>\n The big Google Play Store update is out now, making space for ‘You’ and your gaming skills<\/p>\n Related<\/p>\n This Niagara Launcher feature should be standard across all Android phones<\/p>\n Related<\/p>\n Maybe it\u2019s time to switch services?\n<\/p>\n <\/p>\n I compared Apple Visual Intelligence to Circle to Search, and the winner won’t surprise you<\/p>\n Related<\/p>\n![\"Gemini](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/Gemini-in-Chrome-buy-groceries.jpg\"\/ "\\"Gemini")
<\/p>\n![\"Demo](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/Demo-of-Android-AI-agent-in-Project-Astra_1-scaled.jpg\"\/ "\\"Demo")
<\/p>\n![\"App](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/App-Streaming-on-Chrome-OS.png\"\/ "\\"App")
<\/p>\n![\"COMPUTER](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/COMPUTER_ACCESS_CONTROL-permission.png\"\/ "\\"COMPUTER")
<\/p>\n![\"Demo](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/Demo-of-Android-AI-agent-in-Project-Astra_2-scaled.jpg\"\/ "\\"Demo")
<\/p>\n![\"google](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/google-pixel-watch-4-new-watch-face.jpg\"\/ "\\"google")
<\/a><\/p>\n![\"google](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/09\/google-pixel-watch-4-colors.jpg\"\/ "\\"google")
<\/a><\/p>\n![\"The](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/google-ai-mode-vs-google-search-hero-photo-5-scaled.jpg\"\/ "\\"google")
<\/a><\/p>\n![\"google](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/google-maps-showing-a-route-on-a-smartphone-screen-scaled.jpg\"\/ "\\"google")
<\/a><\/p>\n![\"Google](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/Google-Photos-logo-on-smartphone-Stock-photo.jpg\"\/ "\\"Google"){kind=logo}
<\/a><\/p>\n![\"Close-up](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/09\/google-photos-app-close-up-logo-scaled.jpg\"\/ "\\"google"){kind=logo}
<\/a><\/p>\n![\"Google](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/Google-Messages-logo-on-smartphone-laying-on-table-1.jpg\"\/ "\\"Google"){kind=logo}
<\/a><\/p>\n![\"Google](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/09\/Google-Pixel-10-Pro-XL-Gemini-home.jpg\"\/ "\\"Google")
<\/a><\/p>\n<\/a><\/p>\n![\"Photo](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/09\/Photo-of-a-Pixel-phone-with-flipped-navigation-bar-scaled.jpg\"\/ "\\"Photo")
<\/a><\/p>\n![\"Spotify](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/Spotify-stock-photo-3.jpg\"\/ "\\"Spotify")
<\/a><\/p>\n![\"One](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/One-UI-8.5-logo-on-a-Samsung-Galaxy-phone-scaled.jpg\"\/ "\\"One"){kind=logo}
<\/a><\/p>\n![\"Google](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/Google-Pixel-4-camera-closeup.jpg\"\/ "\\"Google")
<\/a><\/p>\n![\"Samsung](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/09\/Samsung-Galaxy-Ring-Battery-Swelling-Feature-Image.jpg\"\/ "\\"Samsung")
<\/a><\/p>\n![\"samsung](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/samsung-galaxy-watch-sleep-2.jpg\"\/ "\\"samsung")
<\/a><\/p>\n![\"google](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/google_home_devices_mix_1.jpg\"\/ "\\"google")
<\/a><\/p>\n![\"Google's](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/google-nest-hub-second-generation-review-photo-frame-2-scaled.jpg\"\/ "\\"google")
<\/a><\/p>\n![\"google](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/google_home_speaker_1.jpg\"\/ "\\"google")
<\/a><\/p>\n![\"google](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/google-nest-cam-indoor-home-speaker-berry.jpg\"\/ "\\"google")
<\/a><\/p>\n![\"A](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/09\/Android_15_restricted_settings_hero_image.jpg\"\/ "\\"A")
<\/a><\/p>\n![\"Photo](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/Photo-of-the-Android-Bot-wearing-a-jumpsuit-behind-bars-scaled.jpg\"\/ "\\"Photo")
<\/a><\/p>\n![\"Bluetooth](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/Google_Pixel_Buds_Pro_2_audio_sharing_hero_image.jpg\"\/ "\\"Bluetooth")
<\/a><\/p>\n![\"Samsung](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/Samsung-Galaxy-Tab-S11-Ultra-with-keyboard-folio-and-S-Pen.jpg\"\/ "\\"Samsung")
<\/a><\/p>\n![\"Google](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/09\/Google-Play-Store-logo-on-smartphone-stock-photo-2.jpg\"\/ "\\"Google"){kind=logo}
<\/a><\/p>\n![\"google](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/09\/google-pixel-8-pro-niagara-launcher-blank-homescreen-scaled.jpg\"\/ "\\"google")
<\/a><\/p>\n
![\"Apple](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/Apple-Visual-Intelligence-vs-Circle-to-Search-Google-interface.jpg\"\/ "\\"Apple")
<\/a>![\"duolingo](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2025\/10\/duolingo-missions-on-a-smartphone-screen-scaled.jpg\"\/ "\\"duolingo")
<\/a><\/p>\n