rec

Splash is a local inference engine for Apple silicon developed by Inco AI. It serves selected language models to coding agents and clients compatible with the OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages APIs, with streaming, tool calls, JSON Schema output, image input, and inline PDF support. Its runtime uses model-specific fused Metal kernels, weights packed for those kernels, a dedicated DFlash 2 draft model for speculative decoding, and a startup-computed memory plan for context, KV cache, and batching. The served model packages include the target model and its draft model; plain MLX or Transformers checkpoints are not supported. Splash runs locally on macOS, binds to a local HTTP server by default, and can be installed through Homebrew. The project is licensed under Apache-2.0, while model weights retain their own licenses.

oMLX is an open-source LLM inference server for Apple Silicon Macs, managed through a macOS menu-bar application. It serves text, vision, OCR, embedding, and reranking models through compatible API endpoints, with model discovery and management, monitoring, continuous batching, and caching. Its caching system keeps KV cache in a hot in-memory tier and a cold SSD tier, allowing cached context to remain reusable across requests even when the conversation context changes. The project can be installed as a macOS app, through Homebrew, or from source, and includes a CLI for controlling the app-managed server. The repository states that it requires macOS 15 or later, Python 3.11–3.13, and Apple Silicon hardware.

DwarfStar is a self-contained, model-specific native inference engine for running DeepSeek V4 Flash, GLM 5.2, and, on high-memory systems, DeepSeek V4 PRO locally. It combines model loading, prompt rendering, tool calls, persistent KV state, an HTTP server, and a coding agent rather than serving as a general GGUF runner. The engine supports Metal on Macs, CUDA including multi-GPU systems, and ROCm on Strix Halo hardware; it also provides SSD streaming for machines with insufficient RAM, distributed tensor and pipeline parallelism, and server-side micro-batching for multi-user inference. The repository includes tooling and data for GGUF, imatrix, quality, and speed testing, and acknowledges code and design contributions from llama.cpp and GGML.

Reasonix is an open-source AI coding agent built around a local engine and distributed as a self-contained static Go binary. It is designed for DeepSeek, with DeepSeek provided as a preset, while configuration can target OpenAI-compatible endpoints and optionally use separate executor and planner models. The agent can be used through a terminal or desktop app, in a browser, or from an editor through ACP; it provides plan mode, permissions, workspace sandboxing, per-turn checkpoints, and MCP- or extension-based plugins. Its context maintenance is designed around prefix-cache stability, including a stable startup environment summary and pruning of stale tool output before compaction.

Lithos Metal is an open-source LLM inference engine for Apple silicon that generates specialized Metal megakernels and serves models locally for coding agents and other applications. Its compiler decomposes operators into matrix tiles, attention partitions, and recurrent-state slices, records their dependencies, and generates layer-wise fused GPU tasks. The runtime combines mixer megakernels, GPU-resident generation control, pre-encoded Metal indirect command buffers, and DSpark speculative decoding, in which a target-conditioned draft proposes token blocks and the target model verifies and commits the accepted prefix. The server provides OpenAI-compatible Chat Completions and Responses adapters plus an Anthropic Messages adapter, with text generation, tool calls, and SSE streaming; launchers configure clients including OpenCode, Claude Code, Codex, and Hermes. It runs on Apple silicon with macOS 26 or later, is distributed through a Lithos Homebrew tap or source installation, and is licensed under Apache 2.0.