Laya-CoreML

Laya is an open-source, Jev-compatible System-1 decision model package by Convai Innovations for running typed decision questions from Node.js and TypeScript through ONNX Runtime. It accepts a state such as a ticket, email, or JSON object and returns calibrated answers for choice questions with per-option probabilities, ordered score questions with a probability distribution and expected level, or yes/no statements with a probability of true. Questions in one call are batched into a single model run, and the package uses the same request and response shape as the Python reference implementation without requiring Python or PyTorch at runtime. The package downloads and caches ONNX model weights from Hugging Face by default, or can load a local ONNX bundle; it requires Node.js 20 or newer. The JavaScript package is MIT-licensed, while the Laya model weights are published under Apache 2.0.

MLX is an array framework for machine learning on Apple silicon, developed by Apple machine learning research. It provides Python, C++, C, and Swift APIs modeled on NumPy, plus higher-level packages such as mlx.nn and mlx.optimizers with PyTorch-like interfaces. Its composable transformations support automatic differentiation, automatic vectorization, and computation-graph optimization; computations are lazy, graphs are built dynamically, and operations can run on supported CPUs and GPUs using shared unified memory rather than explicit data transfers. MLX is distributed through PyPI and supports model training and deployment workflows including language-model training and generation, LoRA fine-tuning, image generation, and speech recognition.

Laya is an Apache-2.0 Python package and encoder-model decision engine developed by Convai Innovations. It accepts text or JSON state and evaluates typed questions in a single forward pass, returning categorical choices with probabilities and confidence, ordinal scores, or calibrated Boolean probabilities rather than generated text. Its stated uses include routing, ticket triage, prompt-injection detection, moderation, phishing detection, and churn-risk classification. Laya includes English, multilingual, and typed-decisions checkpoints, plus a router that selects a checkpoint using explicit settings, detected script, and best-effort language detection. The checkpoints use reinforcement learning against strictly proper scoring rules (RLCD); the project also provides fine-tuning and temperature-calibration workflows. The multilingual checkpoint is intended for more than 100 languages, while the repository cautions that the base models can perform near chance on some zero-shot typed-decision tasks and that large choice sets are constrained by the option-token budget.

oMLX is an open-source LLM inference server for Apple Silicon Macs, managed through a macOS menu-bar application. It serves text, vision, OCR, embedding, and reranking models through compatible API endpoints, with model discovery and management, monitoring, continuous batching, and caching. Its caching system keeps KV cache in a hot in-memory tier and a cold SSD tier, allowing cached context to remain reusable across requests even when the conversation context changes. The project can be installed as a macOS app, through Homebrew, or from source, and includes a CLI for controlling the app-managed server. The repository states that it requires macOS 15 or later, Python 3.11–3.13, and Apple Silicon hardware.

MTPLX is an open-source native Mac app and command-line tool for running local language models on Apple Silicon. It uses a model's built-in multi-token prediction (MTP) heads to draft several tokens, verifies the block in one batched forward pass, and commits tokens with exact rejection sampling and residual correction, without requiring a separate draft model or changing the model's sampling distribution. It provides local OpenAI-compatible and Anthropic-compatible APIs, native chat, model management, hardware-specific draft-depth tuning, and tooling for building and verifying MTP models. The server can also expose MLX embedding and reranking models. MTPLX requires an M1-or-newer Mac running macOS 14 or later, and the repository is licensed under Apache-2.0.