TurboFieldfare

Swiftlet is an open-source Swift and Metal runtime for running Qwen3-Next and Qwen3.5/3.6 mixture-of-experts models locally on Apple devices, including iPhones. It keeps the models' small dense core in memory and streams routed expert weights from storage on demand, allowing 35B and 80B models to run with comparatively low RAM. The project is available as a Swift package, command-line interface, OpenAI-compatible server, and iOS app; its repository describes the runtime as working end to end and notes that only about 3B parameters are active per token in the supported models.

slotstream is an open-source macOS local-LLM inference engine that runs Qwen3.8-Flash-Next, a 125-billion-parameter mixture-of-experts model, on Apple Silicon Macs whose memory cannot hold the model. Implemented as a single Swift binary using MLX, it keeps the dense trunk resident and reads routed expert weights from an SSD into a shared cache of slots, sizing that cache to available memory instead of loading the full model. It supports prefix caching for follow-up turns and exposes Ollama-compatible and OpenAI-compatible chat APIs, as well as a command-line prompt mode; the documented API subset supports streaming, CORS, and sampling options but not tools, images, JSON-schema output, or logprobs. The project requires macOS 14 or later, Apple Silicon, and roughly 110 GB of free disk space for the model weights. Its installer provides signed release binaries, and the repository can also be built with Swift and the macOS Command Line Tools. The source is MIT-licensed; the model weights are distributed under the Qwen community license.

edge0 is an open-source streaming Mixture-of-Experts inference framework for running sparse MoE models on Apple Silicon Macs. Its MLX backend memory-maps expert weights from SSD storage and loads them on demand, so peak memory depends on the active experts rather than the model's total parameter count. A trained prerouter predicts the next expert routes so storage reads can overlap the forward pass, while Recover-LoRA adapters distill from the full-precision model to compensate for quantization loss; the base weights remain read-only and the adapters stay separate. The framework ships two end-to-end model tiers with bundled 4-bit checkpoints, LoRA adapters, and prerouter heads. It provides a command-line interface and Python API, supports demo, chat, and server modes, and exposes an OpenAI-compatible `/v1/chat/completions` endpoint. The current backend supports macOS on Apple Silicon M1 through M4; CUDA support is identified as planned. The repository is licensed under Apache-2.0.

MTPLX is an open-source native Mac app and command-line tool for running local language models on Apple Silicon. It uses a model's built-in multi-token prediction (MTP) heads to draft several tokens, verifies the block in one batched forward pass, and commits tokens with exact rejection sampling and residual correction, without requiring a separate draft model or changing the model's sampling distribution. It provides local OpenAI-compatible and Anthropic-compatible APIs, native chat, model management, hardware-specific draft-depth tuning, and tooling for building and verifying MTP models. The server can also expose MLX embedding and reranking models. MTPLX requires an M1-or-newer Mac running macOS 14 or later, and the repository is licensed under Apache-2.0.

Splash is a local inference engine for Apple silicon developed by Inco AI. It serves selected language models to coding agents and clients compatible with the OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages APIs, with streaming, tool calls, JSON Schema output, image input, and inline PDF support. Its runtime uses model-specific fused Metal kernels, weights packed for those kernels, a dedicated DFlash 2 draft model for speculative decoding, and a startup-computed memory plan for context, KV cache, and batching. The served model packages include the target model and its draft model; plain MLX or Transformers checkpoints are not supported. Splash runs locally on macOS, binds to a local HTTP server by default, and can be installed through Homebrew. The project is licensed under Apache-2.0, while model weights retain their own licenses.