AI product
Llama is a free, open-source desktop app by the llama.cpp team and Hugging Face that runs open AI models locally on Mac and Windows computers. It provides a system-tray interface for chatting with models, recommends models and configures their llama.cpp settings for the host computer, and downloads models for local use. Models are stored in the shared Hugging Face cache, loaded on demand, and unloaded after an idle period.
Llama also exposes a local API at localhost:9931 using OpenAI-compatible endpoints, with support for streaming, tool calling, structured output, and vision; the page also documents Anthropic-compatible endpoints. Coding agents, editors, chat applications, scripts, and other OpenAI-SDK clients can connect by changing their base URL, without API keys or usage charges. It can share models with llama.cpp and includes command-line installation and serving options.
The app works offline, is distributed for Windows 11 and Mac, and is also available through package managers. The page describes a small native app and says that model files are downloaded separately rather than bundled with the application.
1 use taken from transcripts — each links to the moment in the video.
A free open-source Windows app that runs open AI models on your own PC, living in the system tray, auto-configuring models, and exposing a local OpenAI-compatible API for chat, coding agents, and editors, fully offline.
1 in the library.