Transformers.js

TurboFieldfare is a model-specific Swift and Metal runtime for running the instruction-tuned Gemma 4 26B-A4B model on Apple Silicon Macs, including systems with 8 GB of RAM. It keeps the model’s shared 1.35 GB core and FP16 key-value cache in memory while streaming the expert weights selected for each token from an SSD, allowing inference with about 2 GB of resident weights instead of loading the approximately 14.3 GB model. The model uses MLX affine 4-bit weights with group size 64, an 8-bit router, 4-bit shared and routed experts, and about 3.88 billion active parameters per token. The project includes a native macOS app, command-line interface, Swift runtime library with Metal kernels, decode service, streaming model installer and verifier, and experimental loopback OpenAI-compatible Chat Completions server. These products share a repacked `.gturbo` model directory; the installer streams byte ranges from a pinned Hugging Face revision, repacks them without materializing the source checkpoint, and validates the resulting manifest and file hashes. The app and CLI support instruction and completion generation with optional system guidance but do not execute tools; the server accepts function-tool declarations and returns model-produced tool calls for client authorization and execution. An optional companion vision tower adds image input on M2 or newer Macs, while text-only inference remains available on M1. The package is arm64-only and targets macOS 26, Metal 4, and Swift 6.2 or newer, with approximately 14.3 GB of storage for the text model plus about 1.1 GB for the image pack.

Google's developer API for building applications with Gemini large language models. It provides access to models like Gemini 2.0 Flash and 2.5 Pro, and also supports Gemma open models. Developers use it for tasks such as custom model testing, retrieval, memory, and tool-calling systems.

Transformers is Hugging Face's open-source model-definition framework and Python library for inference and training with pretrained text, computer vision, audio, video, and multimodal machine-learning models. It centralizes model definitions behind a unified API and provides the Pipeline interface, which preprocesses inputs and returns task-specific outputs for applications such as text generation, automatic speech recognition, image classification, and visual question answering. Models can be downloaded and cached from the Hugging Face Hub, and the library supports moving models between PyTorch, JAX, and TensorFlow workflows while exposing model internals for customization. It requires Python 3.10 or later and PyTorch 2.5 or later for the documented installation.

Google AI Edge Gallery is an app for downloading and running Gemma models locally on Android and iOS devices. It supports image description, multilingual audio transcription, on-device function calling, and built-in skills such as games, haiku generation, weather queries, and calendar events. Inference runs on the device, using supported hardware accelerators rather than requiring an API call or sending data to a remote service. Google describes AI Edge as its platform for deploying machine-learning solutions across mobile, web, and embedded applications.

WebLLM is an in-browser large language model inference engine developed by the MLC community. It runs model inference directly in web browsers using WebGPU hardware acceleration, without server-side processing, and can be used as an npm, Yarn, or CDN package for building web applications. Its MLCEngine exposes an OpenAI-compatible chat-completions interface with streaming, JSON-mode structured generation, seeding, and preliminary function-calling support. It supports model families including Llama, Phi, Gemma, Mistral, and Qwen, and can load custom MLC-format models consisting of model artifacts and WebAssembly model libraries. WebLLM provides Web Worker and Service Worker integrations, Chrome-extension examples, browser caching through Cache API, IndexedDB, or OPFS, and optional Subresource Integrity verification for downloaded model artifacts.