Open-source project lets a Safari agent control an iPhone via an ESP32 Bluetooth bridge
Developers have released iPhone Agent Harness, an MIT-licensed open-source project that lets users describe a phone task in Safari. A vision model inspects the screen while a nearby ESP32 board, connected over Bluetooth and Wi-Fi, supplies taps and keyboard input through AssistiveTouch. The demo shows the agent setting an alarm in the iPhone Clock app, though the authors note it is not a benchmark for arbitrary apps or long tasks. The setup requires iOS or iPadOS 27 or later, and firmware is installed through desktop Chrome or Edge. Users choose their own model provider, and a model can still misread the screen, so results must be checked on the phone.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in