Gigatoken

KTransformers is an open-source research framework for efficient large-language-model inference and fine-tuning through heterogeneous CPU-GPU computing. Its user-facing capabilities are provided through the kt-kernel source tree: high-performance inference and supervised fine-tuning (SFT). For mixture-of-experts models, it uses heterogeneous expert scheduling, placing frequently used or “hot” experts on GPUs and other experts on CPUs with optimized AMX, AVX512, or AVX2 kernels and NUMA-aware memory management. It supports CPU-side INT4/INT8 quantized weights, GPU-side GPTQ, native BF16 and FP8 precision, and multiple CPU, GPU, and NPU backends. The SFT capability integrates with LlamaFactory for hybrid CPU-GPU fine-tuning of large MoE models, including LoRA and full-parameter training, with INT4/INT8 quantization options.

TokensBurned is a privacy-first tool that publishes AI coding activity on a GitHub profile through a live SVG card. It collects token counts and model metadata from AI coding harnesses, reduces raw sessions locally, aggregates usage into 15-minute buckets, and serves totals, activity heatmaps, harness/provider/model comparisons, and an optional site-wide rank. It supports native or official integrations for Claude Code, Codex, Gemini CLI, Cline CLI or SDK, and GitHub Copilot CLI, with OTLP or standalone CLI fallbacks for other harnesses. The tool uploads aggregate token counts, harness/provider/model metadata, hashed session identifiers, time buckets, and request counts, while its stated privacy boundary excludes prompts, responses, source code, tool payloads, repository names and paths, transcript files and paths, and API keys. Public display is a separate explicit opt-in tied to a verified GitHub account; users can build cards with selectable layouts, themes, heatmaps, comparisons, and rankings. It is distributed through harness plugins and an npm CLI, installs no cron job, daemon, proxy, or Git synchronization task, and is licensed under the MIT License.

Needle 2 is an open-source, 45-million-parameter foundation model for tool calling, device use, and structured extraction on small devices. Its weights are compressed into a single 14 MB binary and integrated with its inference engine; the repository reports that a full session uses about 28 MB of RAM and performs inference without network access after setup. The model converts text and tool descriptions into schema-constrained JSON calls using a byte-level grammar, attaches a learned confidence score for threshold-based escalation, and can retrieve the top five tools from a larger catalogue. It uses a 256-token sliding window with tools pinned as key-value sinks to bound memory, and its Simple Attention Network combines a Hadamard MLP, grouped-query attention, engram key-value memory, and multi-lane hyper-connections. The Python package supports inference, LoRA fine-tuning, and checkpoint export; the inference engine is fetched from Hugging Face and cached, while the training stack is an optional installation.

An Apache-2.0 C++23/GGML port of SkinTokens and TokenRig for automatic 3D mesh rigging on CPU or Vulkan. It takes a static GLB mesh, uses a Michelangelo point encoder, Qwen3-based TokenRig policy, FSQ expansion, condition encoder, and SkinVAE decoder to predict a skeleton and per-vertex skin weights, then exports a rigged GLB. It can also generate weights for an existing static or animated skeleton, inspect GLB assets, and retarget recognized SOMA30 motion to the 52-joint Mixamo order or to a generated humanoid rig. The project provides a command-line interface, shared library, C11-compatible API, and a Go/WebGL demo; converted GGUF model weights are available separately, with optional Vulkan support.

MTPLX is an open-source native Mac app and command-line tool for running local language models on Apple Silicon. It uses a model's built-in multi-token prediction (MTP) heads to draft several tokens, verifies the block in one batched forward pass, and commits tokens with exact rejection sampling and residual correction, without requiring a separate draft model or changing the model's sampling distribution. It provides local OpenAI-compatible and Anthropic-compatible APIs, native chat, model management, hardware-specific draft-depth tuning, and tooling for building and verifying MTP models. The server can also expose MLX embedding and reranking models. MTPLX requires an M1-or-newer Mac running macOS 14 or later, and the repository is licensed under Apache-2.0.