VisionState

RuView is an open-source Wi-Fi sensing platform from ruvnet that uses Channel State Information (CSI) captured by low-cost ESP32 sensors to turn disturbances in radio waves into spatial data. It detects presence and occupancy through walls, estimates breathing and heart rate without contact, recognizes activities such as walking, sitting, gestures, and falls, and uses RF fingerprinting to identify rooms and environmental changes. The platform also supports camera-free pose estimation, CSI recording and model training, and loading RVF and LoRA profiles. RuView can publish sensing data to Home Assistant through MQTT and integrate with Apple Home, Google Home, Amazon Alexa, and SmartThings through supported bridges and Matter. Its repository also documents local automation, room-level semantic states, and the RuView MetaHarness, an AI-assisted workflow for onboarding, calibration, training, verification, and checking sensing claims.

CarWatch is an open-source, offline in-car AI agent from ThinkOffApp that runs on a Raspberry Pi 5 with a locally hosted language model. It joins chat rooms as a vehicle agent, answers questions from the car's owner's manual using lexical retrieval-augmented generation and page citations, and reads live vehicle and system data while stating when information cannot be sensed. A continuous energy-based voice listener and whisper.cpp provide local speech input without a wake word, with responses played through the car's speakers. The system can monitor OBD and engine data, send departure and arrival messages, trip summaries, and dashcam clips, and expose a phone dashboard for approvals, replies, model selection, and maintenance. The repository describes systemd-managed services for the model server, room agent, voice listener, dashboard, and engine watcher, with Python standard-library code and no cloud subscription required.

LightNav-0 is an open-source generalist embodied-navigation model from the Light Origins Team. It uses a pretrained Qwen3-VL backbone with a single egocentric RGB stream and natural-language instructions to control humanoid, quadruped, wheeled, and aerial robots, transferring across tasks, embodiments, and scenes without per-task or per-benchmark fine-tuning. The model extends the vocabulary with dual-channel pointing tokens and residual vector-quantized action tokens rather than adding navigation-specific prediction heads. At each step, it emits an affordance point representing a feasible local direction or waypoint and an object point representing the goal, followed by three action tokens that decode to ten future SE(2) waypoints for an embodiment-specific low-level controller. Its temporally aware history compressor samples older frames less frequently, pools them more coarsely, and preserves ordering with timestamp tokens while bounding the visual context. The repository includes a released checkpoint and action decoder, command-line prediction, a vLLM-based server with WebSocket streaming, a MuJoCo simulation demo, evaluation harnesses, and a ROS 2 deployment stack with adapters for the Unitree Go2 and LimX TRON 1. The project is released under the Apache License 2.0; the included EVT-Bench is separately licensed CC BY-NC-SA 4.0.

ESP32 iPhone Agent is an open-source AI automation tool that lets a vision model control a real iPhone without installing an app, using a computer, AirPlay mirroring, and an ESP32-S3 board acting as a USB mouse and keyboard. UxPlay receives the AirPlay stream, a GStreamer pipeline converts it into JPEG frames, and the computer sends relative pointer, typing, scrolling, and key commands to the board over Wi-Fi. A planner model creates a short plan from the task and current screen, while an actor model examines each subsequent screenshot and selects actions such as tapping, swiping, drawing, typing, pressing keys, going home, or finishing. The server tracks and recalibrates the relative pointer, checks whether the screen changed after each action, and can retry or re-plan after missed actions, repeated actions, dropped links, or invalid model responses. The project provides a Python/FastAPI server and browser-based interface with live phone control, logs, screenshots with action overlays, manual controls, calibration, and an emergency stop. It runs on macOS or Linux with an OpenAI-compatible vision model, either remote or local; the phone requires AssistiveTouch but no Mac, developer account, jailbreak, or installed app. The repository is licensed under MIT.

A text-recognition API in Apple's Vision framework that detects and recognizes text in images and video and returns recognized strings, confidence scores, and VNTextObservation bounding boxes (normalized coordinates). Provided by Apple as part of the Vision framework (API types include VNRecognizeTextRequest and VNTextObservation), it runs on iOS, iPadOS, macOS and visionOS and is distributed to developers via Xcode and the Apple Developer Documentation; outputs can be converted to screen/tap coordinates for UI automation or accessibility uses.