FreeSolo Flash

Soup is an open-source Python CLI for fine-tuning and post-training large language models from a YAML configuration and a single training command. It supports supervised fine-tuning, LoRA, QLoRA, preference optimization, quantization, automatic batch-size and GPU detection, and local training without a cloud service. Its optional layer-streaming mode keeps the frozen base model out of VRAM and feeds it to the GPU one decoder layer at a time, reducing memory use for low-VRAM systems. The project describes training an 8B model on a 4 GB laptop GPU with this mode; layer streaming is marked beta.

Unsloth is an open-source local desktop and web tool for running, training, fine-tuning, and deploying language, diffusion, embedding, audio, and vision models. It supports Windows, macOS, Linux, WSL, multi-GPU setups, NVIDIA, AMD, and Intel GPUs, CPUs, and the Vulkan backend. The project is distributed as Unsloth Desktop, Unsloth Studio, and Unsloth Core. It can run local models, provide an OpenAI-compatible API, connect models to agents and MCP tools, perform search and RAG, and offer remote or LAN access. Its training functions cover reinforcement learning, LoRA, QLoRA, full fine-tuning, pretraining, GRPO, DPO, and FP8, with export formats including GGUF, NVFP4, and FP8. The repository claims that its fine-tuning workflows use two times less training time and 70% less VRAM.

Deltafin is a native binary for running the full, unpruned Kimi K3 mixture-of-experts model on consumer hardware, along with an OpenAI-compatible API server for local chat and coding agents. Its demonstration approach keeps the model's attention spine on local disk and fetches routed experts from Hugging Face's CDN as needed, allowing the full model to run on a 64-gigabyte MacBook rather than loading the entire expert bank into memory. The project emphasizes preserving Kimi K3's original expert weights and having K3 verify every token; it is presented as an experiment in pushing large-model inference on local hardware rather than as a practical chat setup.

Soup CLI is a command-line toolkit for training, fine-tuning, evaluating and managing large language models (notably Llama-3.1-8B) with emphasis on running on consumer GPUs. It implements streaming/quantization (NF4), multiple training algorithms and evaluation suites, VRAM probing and model shipping/verification features. The project is published at trysoup.dev and installs without requiring PyTorch, supporting Python 3.10–3.12.