Google wants Gemini to do the boring clicking for you. As of this week, the Gemini app for Mac can take a plain English request, open the apps it needs, type, fill out forms and move through websites until the job is done. You watch it happen in a small live preview, and your own mouse never moves.
The feature is called computer use, and Google is fairly late to it. OpenAI, Anthropic and Meta already ship agents that can operate a desktop. What’s interesting is the timing. The same week Gemini learned to drive a Mac, Bloomberg spotted Apple hiring someone to build “an AI agent that can use a computer,” and Apple pushed a macOS update that makes it much harder for outside agents to take full control. The fight over who gets to run your computer just got a lot more crowded.
The short version
- What: Gemini for Mac can now operate local files, desktop apps and websites on your behalf
- Where: US only, English only, for now
- Who: Google AI Plus, Pro and Ultra subscribers, 18 and older, on personal accounts
- Hardware: Apple Silicon Mac running macOS Sequoia 15.0 or later
- Safety net: you pick which apps it can touch, watch every step, pause anytime and approve key actions
- Meanwhile: Apple is building its own Mac agent and tightening permissions for everyone else
How Gemini’s computer use works
The main design choice is where the agent does its work. Instead of hijacking your screen, Gemini runs inside an isolated background window, and a live preview of what it’s doing appears in your chat. “Your main screen and cursor remain completely free, so you can keep typing, designing, or taking calls without interruption,” Google says in its announcement.
Based on Google’s post and early hands-on reports, using it looks like this:
- Open the Gemini app for Mac and type your request in the chat, adding
@Computer useto call the feature. - Choose which apps Gemini may access. New desktop connectors, including Blender and Autodesk Fusion, show up under Customize > Connectors, according to 9to5Google.
- Let it work in the background window while you keep doing your own thing.
- Watch the live preview and pause it whenever something looks off.
- Approve key actions before they happen. Gemini also stops for logins and payments.
That approval step is the important one, and it’s also the vaguest. Google says Gemini asks before “key actions,” but it doesn’t define which actions count as key. Until that’s spelled out, assume anything involving money, sending messages or deleting files deserves a close look.
What Google thinks you’ll use it for
Google’s examples lean heavily toward chores that normally take dozens of clicks. Its own promo shows someone asking Gemini to take a kid’s butterfly sketch, model it and rig it in Blender.
| Google’s example | What Gemini actually does | Touches the web? |
|---|---|---|
| Creative workflows | Batch color-grades photos in design software or builds 3D scenes | No |
| Shipping quotes | Pulls weights, dimensions and addresses from local PDFs into multi-field forms | Yes |
| Presentation decks | Grabs metrics and screenshots from live dashboards and builds a Google Slides deck | Yes |
| Resale listings | Analyzes clothing photos, researches prices, logs details and stages listings | Yes |
Three of the four involve websites, which raises a question Google hasn’t answered yet: how Gemini identifies itself to the sites it visits. Retailers and banks are already split on agents. Some build special lanes for them, others block them, and several big US banks now say you’re responsible for anything an AI agent does with your login. Worth keeping in mind before you point Gemini at an account that holds money.
Google’s year of building a Mac agent, one piece at a time
Computer use didn’t come out of nowhere. Google has used the Mac as a testing ground for desktop agents most of this year, adding a capability every month or two.
Look closely at the last two entries and you’ll see Google’s pricing strategy. The task agent that works with your files and connected Google services went free in the US two days before computer use launched. The part that actually drives your screen and other programs starts at a paid tier. Google has split its desktop agent into a free layer that handles your stuff and a paid layer that operates your machine.
Who can turn it on today
| Requirement | Details |
|---|---|
| Computer | Apple Silicon Mac (M-series). Intel Macs are left out |
| Operating system | macOS Sequoia 15.0 or newer |
| Subscription | Google AI Plus, Pro or Ultra. Free accounts don’t get it |
| Account type | Personal Google account, age 18+. Work and school accounts are excluded |
| Region and language | United States, English only. No date given for other markets |
| Windows | Not included yet, even though Gemini has a Windows app |
Apple wants an agent of its own
The job listing Bloomberg found reads like a product pitch. Apple is looking for a senior engineering manager to oversee an “AI agent that can use a computer.”
“Imagine an AI agent that can use a computer the way a capable person does. It browses the web, works across apps, runs commands, writes and debugs code, and calls services, and it chooses the right tool for each step. It does all this privately and with the polish people expect from Apple.”
Apple job listing, as reported by Bloomberg
Don’t hold your breath. Bloomberg says Apple has looked at launching it as soon as the end of next year, but the work “remains in the early stages.” It also fits a broader push. Bloomberg reported earlier this month that Apple wants to turn Siri into an autonomous agent that gets things done for you. Microsoft is on the same path, and is already rebuilding Windows around AI agents.
And it’s making outside agents work harder for access
Here’s where it gets awkward for Google, Meta and everyone else. While Apple builds its own agent, macOS 27.3 raises the bar for third-party ones. Apple told developers in October that users who “genuinely wish to grant an app this extraordinary level of access” should only be able to do it “with very explicit user action.” The update follows through:
- The old all-in-one setting is split. Early beta coverage describes a broad “System & App Protection Override” permission for apps that need to control other software, separate from a narrower Accessibility permission.
- Louder warnings. macOS will now discourage people from handing full control to outside agents.
- Accessibility goes back to its purpose. Apple is sharply restricting developers’ use of accessibility tools on Macs and adding a new set of permissions built for agents.
- Old grants don’t vanish. Permissions you already approved can stay active after the update.
Apple has a fresh reason to do this. Reuters found this week that Meta’s Muse agent read Signal messages on a Mac by combining Accessibility and screenshot permissions, without using any official Signal connection. Apple says its update closes that pathway.
Worth knowing: Apple’s caution cuts both ways. Tighter permissions protect you from overreaching agents, but they also make life harder for rivals right as Apple prepares its own. Expect the “privacy versus competition” argument to get loud once Apple’s agent ships.
Before you let Gemini drive
Computer use is genuinely handy for repetitive work, and Google’s background window design is a smart way to keep the agent from fighting you for the cursor. A few habits will keep it from becoming a headache:
- Start with low-stakes jobs. Renaming files, sorting folders or filling a form you’ll review yourself are perfect first tests.
- Grant apps one at a time. Only connect the apps a task needs, and remove them afterward if you won’t use them again soon.
- Actually watch the preview the first few times. You’ll learn quickly where it gets confused.
- Keep it away from banking and shopping logins until Google explains what “key actions” covers and how it handles sites that restrict agents.
- Review your Mac permissions after macOS 27.3. Open System Settings > Privacy & Security and check which apps still hold Accessibility, Screen Recording and Full Disk Access.
- Remember what you share. Anything Gemini sees while working may end up in your chat history, and Google recently changed how long it keeps “History Off” chats.
A year ago, an AI that could click through your apps was a demo. Now Google, Meta, OpenAI and Anthropic all sell one, and Apple is hiring to build its own. The real question is who controls the permissions screen, and on the Mac, that’s still Apple.
About this article: GeekBlog covers U.S. technology news, AI, phones, smartwatches and gaming. Every story is written and checked under our Editorial Policy. Spotted a mistake or have a story tip? Contact our editors.

