Lithos Metal

MTPLX is an open-source native Mac app and command-line tool for running local language models on Apple Silicon. It uses a model's built-in multi-token prediction (MTP) heads to draft several tokens, verifies the block in one batched forward pass, and commits tokens with exact rejection sampling and residual correction, without requiring a separate draft model or changing the model's sampling distribution. It provides local OpenAI-compatible and Anthropic-compatible APIs, native chat, model management, hardware-specific draft-depth tuning, and tooling for building and verifying MTP models. The server can also expose MLX embedding and reranking models. MTPLX requires an M1-or-newer Mac running macOS 14 or later, and the repository is licensed under Apache-2.0.

Splash is a local inference engine for Apple silicon developed by Inco AI. It serves selected language models to coding agents and clients compatible with the OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages APIs, with streaming, tool calls, JSON Schema output, image input, and inline PDF support. Its runtime uses model-specific fused Metal kernels, weights packed for those kernels, a dedicated DFlash 2 draft model for speculative decoding, and a startup-computed memory plan for context, KV cache, and batching. The served model packages include the target model and its draft model; plain MLX or Transformers checkpoints are not supported. Splash runs locally on macOS, binds to a local HTTP server by default, and can be installed through Homebrew. The project is licensed under Apache-2.0, while model weights retain their own licenses.

MLX is an array framework for machine learning on Apple silicon, developed by Apple machine learning research. It provides Python, C++, C, and Swift APIs modeled on NumPy, plus higher-level packages such as mlx.nn and mlx.optimizers with PyTorch-like interfaces. Its composable transformations support automatic differentiation, automatic vectorization, and computation-graph optimization; computations are lazy, graphs are built dynamically, and operations can run on supported CPUs and GPUs using shared unified memory rather than explicit data transfers. MLX is distributed through PyPI and supports model training and deployment workflows including language-model training and generation, LoRA fine-tuning, image generation, and speech recognition.

oMLX is an open-source LLM inference server for Apple Silicon Macs, managed through a macOS menu-bar application. It serves text, vision, OCR, embedding, and reranking models through compatible API endpoints, with model discovery and management, monitoring, continuous batching, and caching. Its caching system keeps KV cache in a hot in-memory tier and a cold SSD tier, allowing cached context to remain reusable across requests even when the conversation context changes. The project can be installed as a macOS app, through Homebrew, or from source, and includes a CLI for controlling the app-managed server. The repository states that it requires macOS 15 or later, Python 3.11–3.13, and Apple Silicon hardware.

h3-metal is a native plain-C and Metal inference engine for the MiniMax H3 video model on Apple Silicon Macs. It provides a command-line interface and Iris-style interactive session for prompt-to-video and prompt-to-audio generation, including synchronized stereo audio, first- and last-frame conditioning, and ordered Ref2VA image, video, and audio references. The CLI can inspect model layout and the selected Metal device, run prompt-to-media jobs, and retain exact BF16 prompt conditioning, the prepared diffusion transformer, and video decoder in memory so repeated prompts with different seeds avoid reloading and re-encoding them. Configurable controls include denoising passes, transition reuse, transformer layers, frame dimensions, seeds, and output files. The project is developed through incremental vertical slices covering deterministic host and model metadata, portable Metal block parity, prompt encoding, media generation, conditioning, and H3-specific Metal performance and memory optimization.