WARP

edge0 is an open-source streaming Mixture-of-Experts inference framework for running sparse MoE models on Apple Silicon Macs. Its MLX backend memory-maps expert weights from SSD storage and loads them on demand, so peak memory depends on the active experts rather than the model's total parameter count. A trained prerouter predicts the next expert routes so storage reads can overlap the forward pass, while Recover-LoRA adapters distill from the full-precision model to compensate for quantization loss; the base weights remain read-only and the adapters stay separate. The framework ships two end-to-end model tiers with bundled 4-bit checkpoints, LoRA adapters, and prerouter heads. It provides a command-line interface and Python API, supports demo, chat, and server modes, and exposes an OpenAI-compatible `/v1/chat/completions` endpoint. The current backend supports macOS on Apple Silicon M1 through M4; CUDA support is identified as planned. The repository is licensed under Apache-2.0.

MoonEP is an expert-parallelism communication library from Moonshot AI for mixture-of-experts training and inference. It plans a small number of dynamic redundant experts from the current router outputs, prefetches their weights before expert computation, and reduces the resulting gradients back to the experts' home ranks during the backward pass. Its near-optimal GPU planning kernel, fused permute/unpermute operations, and zero-copy dispatch send tokens directly to expert-grouped positions on remote ranks and return buffer views to computation. The library uses a fixed S × K token buffer per rank, where S is the input-token count and K is the routed top-k, keeping communication and computation shapes static across layers. The repository documents support for NVIDIA GPUs and lists Zhenwu PPU support as under review.

Colibrì is a pure-C inference engine and open research platform for running large mixture-of-experts models on consumer and heterogeneous hardware without engine dependencies. It treats VRAM, system RAM, and storage as a unified multitier inference hierarchy, streaming expert weights from disk and coordinating placement, storage I/O, scheduling, kernels, speculation, and CPU/GPU overlap. The project provides chat, server, and web front ends, with support for CPU, CUDA, Metal, and dual-SSD streaming; its design rules prohibit silently changing model precision or router semantics when fast memory is insufficient.

APEX is an open, verification-first RTL design for an LLM-inference tile developed with Sigmantic AI. It implements one transformer decoder layer—including attention, softmax, RMSNorm, RoPE, SwiGLU, residual processing, and KV-cache compression—and runs Qwen2.5-0.5B through the verified pipeline on FPGA hardware. The KV-cache codec sits in the datapath: cached keys and values are compressed as they are produced and decompressed when consumed, while an importance unit allocates precision according to which parts of the context matter. Each RTL block is checked against an executable NumPy golden model with bit-exact comparisons and mutation-tested testbenches. The repository covers the inference tile rather than a complete system; DRAM control, PCIe, and NoC are out of scope.

waggle is an open-source reference layer for handing off artifacts between AI agents. It replaces pasted files with compact, approximately 30-byte tokens that resolve into an agent-specific view while preserving attribution and recording which parts were read. The token travels without automatically expanding the underlying artifact. Resolve, read, and search operations return only the requested projection or slice within byte budgets, and corrections can propagate to token holders. waggle is MCP-native and runs as an MCP server for agent tools.