ESP32 iPhone Agent

Phone Harness is an open-source harness that lets AI agents such as Claude Code, Codex, or other LLMs control a real iPhone or Android phone without a jailbreak, injected app, Xcode, WebDriverAgent, or on-device installation. It exposes helpers for opening apps, reading screen content, tapping, scrolling, typing, and verifying actions. On iPhone, it uses the macOS iPhone Mirroring window as the transport: screenshots are captured from the window, Apple's Vision framework performs OCR and returns tap-ready coordinates, and HID-level CGEvents provide taps, long presses, drags, flicks, scrolling, Unicode typing, and shortcuts. On Android, it uses ADB over USB or Wi-Fi, combining screenshots with the phone's accessibility tree for text and element bounding boxes. Actions can be verified by capturing the screen again, with the screenshot serving as the ground truth rather than relying on a DOM. Each invocation is self-contained rather than relying on a daemon.

Port22 is a tool for remotely running and controlling code-focused AI agents. It uses a Mac agent paired with a phone to drive agents such as Claude Code, Codex, OpenCode, and Aider, providing live transcripts, in-phone approval, LAN support, and an end-to-end encrypted relay.

Xcode MCP is an MCP tool that lets an AI agent create an iOS app in VS Code, open and interact with the app, and capture screenshots for a feedback loop.

Agent S is an open-source computer-use agent framework from Simular. It accepts natural-language tasks, observes a real desktop through screen and accessibility signals, and operates ordinary applications by clicking, typing, scrolling, and executing optional local code rather than relying on per-application APIs or scripts. The project has progressed through three generations: accessibility-tree grounding with an LLM, a vision-based manager-worker architecture using multiple foundation models, and Agent S3, which scales computer-use attempts through parallel rollouts and selection. Its workflow can combine model-driven exploration with reusable automation code, replay saved skills, and recover when the interface changes. The framework supports macOS, Windows, and Linux, works with models from OpenAI, Anthropic, Gemini, OpenRouter, vLLM, and open-weight providers, and is distributed through the `gui-agents` Python package. An optional local coding environment can execute Python and Bash with the current user's permissions, so the repository warns that this capability should be used only in trusted environments.