mini-AGI

MiniMind is an open-source tiny large-language-model project and end-to-end training tutorial developed by Jingyao Gong. Its main dense model is approximately 64M parameters, and the repository provides the model architecture, tokenizer, datasets, inference code, and training pipeline. The pipeline covers pretraining, supervised fine-tuning, hand-written LoRA, DPO, PPO, GRPO, CISPO, model distillation, tool calling, adaptive thinking, and agentic reinforcement learning. Core algorithms are implemented directly with native PyTorch rather than relying on high-level abstractions from third-party training libraries. Its agentic reinforcement-learning path performs multi-turn rollouts, executes generated tool calls, appends tool observations to the context, calculates trajectory-level rewards, and updates the policy; rollout can use local PyTorch generation or an SGLang server. MiniMind supports dense and mixture-of-experts variants, single- and multi-GPU training, YaRN-based RoPE length extrapolation, Transformers-format models, and inference through llama.cpp, vLLM, or Ollama. The repository also includes a Streamlit chat interface and a lightweight OpenAI-compatible API server with tool-call and reasoning fields. It is released under the Apache License 2.0.

AirLLM is an open-source Python library for running large language model inference on GPUs with limited memory. It reduces GPU memory use by loading and processing one model layer at a time instead of holding the entire model in GPU memory; for sparse mixture-of-experts models, it streams individual experts as needed. The project is designed to run large models without requiring quantization, distillation, or pruning, and provides a pip-installable package with support for multiple model families, CPU inference, macOS, and optional model compression and quantization features.

NVIDIA DGX Spark is a compact AI workstation from NVIDIA for local development, fine-tuning, and inference of generative-AI models. It is built around the NVIDIA GB10 Grace Blackwell Superchip, with 128 GB of unified memory and up to 1 petaflop of FP4 AI performance, and runs NVIDIA DGX OS.

llama.cpp is a C/C++ implementation for running large language and vision-language model inference locally or in the cloud, including on CPUs, GPUs, laptops, Apple silicon devices, and small computers such as Raspberry Pis. The project is built on the ggml library and uses integer quantization from 1.5-bit through 8-bit to reduce memory use and accelerate inference, with CPU, CUDA, HIP, Metal, Vulkan, SYCL, OpenCL, and other hardware backends. It supports hybrid CPU-GPU inference for models larger than available VRAM and includes command-line tools, a built-in web interface, and an OpenAI-compatible REST API server. The project is distributed as source code, Docker images, and pre-built binaries, with installation instructions at llama.app.

DwarfStar is a self-contained, model-specific native inference engine for running DeepSeek V4 Flash, GLM 5.2, and, on high-memory systems, DeepSeek V4 PRO locally. It combines model loading, prompt rendering, tool calls, persistent KV state, an HTTP server, and a coding agent rather than serving as a general GGUF runner. The engine supports Metal on Macs, CUDA including multi-GPU systems, and ROCm on Strix Halo hardware; it also provides SSD streaming for machines with insufficient RAM, distributed tensor and pipeline parallelism, and server-side micro-batching for multi-user inference. The repository includes tooling and data for GGUF, imatrix, quality, and speed testing, and acknowledges code and design contributions from llama.cpp and GGML.