Anan iPhone Use

ESP32 iPhone Agent is an open-source AI automation tool that lets a vision model control a real iPhone without installing an app, using a computer, AirPlay mirroring, and an ESP32-S3 board acting as a USB mouse and keyboard. UxPlay receives the AirPlay stream, a GStreamer pipeline converts it into JPEG frames, and the computer sends relative pointer, typing, scrolling, and key commands to the board over Wi-Fi. A planner model creates a short plan from the task and current screen, while an actor model examines each subsequent screenshot and selects actions such as tapping, swiping, drawing, typing, pressing keys, going home, or finishing. The server tracks and recalibrates the relative pointer, checks whether the screen changed after each action, and can retry or re-plan after missed actions, repeated actions, dropped links, or invalid model responses. The project provides a Python/FastAPI server and browser-based interface with live phone control, logs, screenshots with action overlays, manual controls, calibration, and an emergency stop. It runs on macOS or Linux with an OpenAI-compatible vision model, either remote or local; the phone requires AssistiveTouch but no Mac, developer account, jailbreak, or installed app. The repository is licensed under MIT.

Mobile MCP is an open-source Model Context Protocol server from Mobile Next for automating and scraping native iOS and Android applications on simulators, emulators, and connected real devices. It exposes a platform-agnostic set of MCP tools for device management, app installation and control, taps, swipes, text input, screenshots, screen recording, device logs, crash reports, deep links, orientation, location, and clipboard operations. The server primarily drives applications through native accessibility trees and structured UI-element data, falling back to screenshots and coordinate-based actions when required. It can run locally over stdio or as a Streamable HTTP server, and can use Mobile Next Cloud to reserve remote physical devices through the same tool interface. It is distributed as the `@mobilenext/mobile-mcp` npm package and requires platform tooling such as Xcode command-line tools or the Android SDK for device access.

Codex iOS Assistant is an open-source system for controlling and inspecting an iPhone from Codex running on a Mac. Its `iphone` CLI sends private commands through iMessage to an iOS Shortcut; the Shortcut performs native iOS actions and returns device context through an authenticated HTTPS response routed via Cloudflare Tunnel to a receiver on the Mac. A per-user macOS LaunchAgent runs the Messages sender outside the Codex sandbox over a Unix socket. The system can read visible screen text, save screenshots, access or replace the clipboard, manage alarms, open specified apps, control timers, the flashlight, and Low Power Mode, place calls, and read Mac Contacts and the Mac Messages database. Message composition creates a draft for review; the CLI does not send ordinary messages, make purchases, install apps, order food, or request rides. It requires macOS 14 or newer, Python 3.11 or newer, Messages, Xcode Command Line Tools, an iPhone reachable at a configured iMessage address, iCloud sync for Shortcuts, and a Cloudflare-managed domain. Installation uses the repository scripts and creates an iOS Shortcut containing 95 native Shortcuts actions, with a manually configured iPhone message automation.

Xcode MCP is an MCP tool that lets an AI agent create an iOS app in VS Code, open and interact with the app, and capture screenshots for a feedback loop.