Apple researchers train an 8B agent on live iOS apps with online reinforcement learning

A team of Apple researchers has published a method for teaching a relatively small AI agent to operate iPhone apps by letting it practise on live software rather than copying recorded demonstrations. The paper, by Peitong Duan, Khashayar Gatmiry, Jeffrey Nichols, Samy Bengio and Emmanuel Abbe, was posted to arXiv on October 29 and is also listed on Apple's machine learning research site. It describes research work, not a feature in any Apple product.

How the agent is built and trained

The authors argue that frontier vision-language models navigate interfaces well but are too costly for routine use, while small models trained by supervised fine-tuning or offline reinforcement learning tend to memorise fixed paths and break when something goes wrong. Their answer is online reinforcement learning on running iOS apps.

The agent has two parts. A planner, based on Qwen3-VL-8B-Instruct and first warmed up with supervised data, decides what to do next; an internal 3B grounding model turns that decision into taps and text entry. A language model acting as judge scores every step, using Gemini-3.5-Flash, and Gemini-3.1-Pro checks whether the task was completed. Training uses PPO with step-level advantages, plus a variant the team calls GiGPPO, which steadies value estimates by borrowing rewards from visually similar screens seen in parallel runs. The infrastructure runs up to 512 virtual iOS devices at once.

What the numbers show

On UINavBench, 116 tasks across 36 real iOS apps, the best PPO checkpoint succeeds 32.4% of the time, against 16.7% for the supervised baseline. Gemini-3.5-Flash, tested with the same agent setup, scores 28.4% and Gemini-3.1-Pro 32.4%, which is the basis for the paper's claim that the agent beats the former and ties the latter on real apps. On AndroidWorld, which the agent never trained on, success rises from 4.7% to 32.8%, close to a sevenfold gain.

Limits

The picture is less favourable elsewhere. On iOSWorld, a set of synthetic apps, the best RL result is 9.8%, below Gemini-3.5-Flash at 12.9% and far below Gemini-3.1-Pro at 39.5%; the authors call this performance modest. On AndroidWorld the Gemini models still score around 58% to 61%. The grounder only handles directional swipes, so swipes are passed to Gemini-3.5-Flash, and simulated users are also played by Gemini. The paper does not say whether code or weights will be released.