AI product Open source
OpenVLHarness is an open-source framework for turning multimodal language models into visual agents without fine-tuning their backbone weights. The model selects tools and reads their evidence through nine task-level capabilities: visual grounding, OCR, depth estimation, camera-trajectory estimation, image zoom, text search, reverse image search, webpage visits, and a Python coding agent. Its multimodal memory keeps images, boxes, masks, depth maps, numerical data, and web references addressable across a question, allowing later tools or generated Python code to combine earlier results—for example, locating objects, estimating their spatial relationship, or searching for and verifying an unfamiliar object. The harness includes model presets, a configurable tool router, evaluation utilities, and support for local or API-served models; the main code is released under the MIT License, while some evaluation, visualization, model, service, and dataset components have separate licenses. The local Python coding executor runs with the agent host's permissions and network and filesystem access, so it should be isolated when handling untrusted workloads.
1 use taken from transcripts — each links to the moment in the video.
A framework that gives vision language models tools to inspect images without retraining their weights, enabling actions like locating objects or reading text within the image.
1 in the library.