Gemini gets hands on your Mac: it can operate the computer and tidy your files
Google says the macOS Gemini app can now use your computer, completing multi-step tasks across local files, desktop apps and the web. The same day, the Gemini API added Tool Retrieval so agents can carry hundreds of tools while spending fewer tokens.
What it can do
In Google's framing, Gemini can now “get multi-step tasks done across your local files, desktop apps, and the web.” The examples skew toward office work: organising files, filling forms, assembling information scattered across several apps into one result. In effect, browser automation has moved into the operating system.
The API side of the same move
Google DeepMind also launched a preview of Tool Retrieval in the Gemini API: when an agent has access to hundreds of tools, it retrieves only the relevant definitions instead of loading every description into context. Tool-definition bloat is one of the main cost drivers for agents today, and this feature aims squarely at that cost structure.
Three preconditions for desktop agents
First, permissions: apps must be willing to expose operable interfaces. Second, reversibility: users need to see and interrupt an action while it runs. Third, failure handling: whether an agent rolls back or pushes on when step four of six fails determines whether it is usable for real work. On consumer products, all three are currently only just adequate.
The competitive picture
Apple is reportedly hiring engineers for its own Mac agent, and Meta's Muse is already shipping on hardware. The desktop is becoming the first real battleground for agents for an obvious reason: plenty of applications, tasks with real value density, and no mobile-grade sandbox.
The quiet risk
Letting a model act on your filesystem is far riskier than letting it write text. A wrong deletion, message or commit is not undoable. That is exactly why real-world usage of these features usually lands well below the polish of the launch demo.