Swiftlet

slotstream is an open-source macOS local-LLM inference engine that runs Qwen3.8-Flash-Next, a 125-billion-parameter mixture-of-experts model, on Apple Silicon Macs whose memory cannot hold the model. Implemented as a single Swift binary using MLX, it keeps the dense trunk resident and reads routed expert weights from an SSD into a shared cache of slots, sizing that cache to available memory instead of loading the full model. It supports prefix caching for follow-up turns and exposes Ollama-compatible and OpenAI-compatible chat APIs, as well as a command-line prompt mode; the documented API subset supports streaming, CORS, and sampling options but not tools, images, JSON-schema output, or logprobs. The project requires macOS 14 or later, Apple Silicon, and roughly 110 GB of free disk space for the model weights. Its installer provides signed release binaries, and the repository can also be built with Swift and the macOS Command Line Tools. The source is MIT-licensed; the model weights are distributed under the Qwen community license.

TurboFieldfare is a model-specific Swift and Metal runtime for running the instruction-tuned Gemma 4 26B-A4B model on Apple Silicon Macs, including systems with 8 GB of RAM. It keeps the model’s shared 1.35 GB core and FP16 key-value cache in memory while streaming the expert weights selected for each token from an SSD, allowing inference with about 2 GB of resident weights instead of loading the approximately 14.3 GB model. The model uses MLX affine 4-bit weights with group size 64, an 8-bit router, 4-bit shared and routed experts, and about 3.88 billion active parameters per token. The project includes a native macOS app, command-line interface, Swift runtime library with Metal kernels, decode service, streaming model installer and verifier, and experimental loopback OpenAI-compatible Chat Completions server. These products share a repacked `.gturbo` model directory; the installer streams byte ranges from a pinned Hugging Face revision, repacks them without materializing the source checkpoint, and validates the resulting manifest and file hashes. The app and CLI support instruction and completion generation with optional system guidance but do not execute tools; the server accepts function-tool declarations and returns model-produced tool calls for client authorization and execution. An optional companion vision tower adds image input on M2 or newer Macs, while text-only inference remains available on M1. The package is arm64-only and targets macOS 26, Metal 4, and Swift 6.2 or newer, with approximately 14.3 GB of storage for the text model plus about 1.1 GB for the image pack.

Splash is a local inference engine for Apple silicon developed by Inco AI. It serves selected language models to coding agents and clients compatible with the OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages APIs, with streaming, tool calls, JSON Schema output, image input, and inline PDF support. Its runtime uses model-specific fused Metal kernels, weights packed for those kernels, a dedicated DFlash 2 draft model for speculative decoding, and a startup-computed memory plan for context, KV cache, and batching. The served model packages include the target model and its draft model; plain MLX or Transformers checkpoints are not supported. Splash runs locally on macOS, binds to a local HTTP server by default, and can be installed through Homebrew. The project is licensed under Apache-2.0, while model weights retain their own licenses.

edge0 is an open-source streaming Mixture-of-Experts inference framework for running sparse MoE models on Apple Silicon Macs. Its MLX backend memory-maps expert weights from SSD storage and loads them on demand, so peak memory depends on the active experts rather than the model's total parameter count. A trained prerouter predicts the next expert routes so storage reads can overlap the forward pass, while Recover-LoRA adapters distill from the full-precision model to compensate for quantization loss; the base weights remain read-only and the adapters stay separate. The framework ships two end-to-end model tiers with bundled 4-bit checkpoints, LoRA adapters, and prerouter heads. It provides a command-line interface and Python API, supports demo, chat, and server modes, and exposes an OpenAI-compatible `/v1/chat/completions` endpoint. The current backend supports macOS on Apple Silicon M1 through M4; CUDA support is identified as planned. The repository is licensed under Apache-2.0.

Qwen Scribe is a private, local transcription and system-wide dictation application for Apple Silicon Macs. It runs Qwen3-ASR through mlx-qwen3-asr on the Mac's Metal GPU, keeping audio and transcripts on the computer without an account, API key, or cloud service. Users can drag audio or video into a local web interface, transcribe with automatic or forced language selection, vocabulary hints, word timestamps, and SRT export, and search or export local transcript history. A menu-bar helper provides push-to-talk dictation in text fields, displaying listening and transcription states in a non-focus-stealing HUD before pasting the result at the cursor. The project requires macOS 14 or newer, Apple Silicon, Python 3.12 or newer, and ffmpeg for non-WAV audio and video. It offers Qwen3-ASR 1.7B and 0.6B model options; model weights are downloaded separately and require roughly 3.4 GB or 1.2 GB of unified memory respectively, in addition to application overhead. It is distributed as a downloadable beta Mac app or can be built from source; the repository describes the project as version 0.2.1-beta.1.