AI product Open source
MiniMind is an open-source tiny large-language-model project and end-to-end training tutorial developed by Jingyao Gong. Its main dense model is approximately 64M parameters, and the repository provides the model architecture, tokenizer, datasets, inference code, and training pipeline.
The pipeline covers pretraining, supervised fine-tuning, hand-written LoRA, DPO, PPO, GRPO, CISPO, model distillation, tool calling, adaptive thinking, and agentic reinforcement learning. Core algorithms are implemented directly with native PyTorch rather than relying on high-level abstractions from third-party training libraries. Its agentic reinforcement-learning path performs multi-turn rollouts, executes generated tool calls, appends tool observations to the context, calculates trajectory-level rewards, and updates the policy; rollout can use local PyTorch generation or an SGLang server.
MiniMind supports dense and mixture-of-experts variants, single- and multi-GPU training, YaRN-based RoPE length extrapolation, Transformers-format models, and inference through llama.cpp, vLLM, or Ollama. The repository also includes a Streamlit chat interface and a lightweight OpenAI-compatible API server with tool-call and reasoning fields. It is released under the Apache License 2.0.
2 uses taken from transcripts — each links to the moment in the video.
An open-source project for training a tiny language model from scratch in about two hours on modest hardware. It includes model code, a tokenizer, and stages such as pre-training, supervised fine-tuning, preference optimization, tool use, and distillation.
An open-source project for training a small language model from scratch in about two hours on modest hardware. It includes model code, a tokenizer, and training stages such as pretraining, supervised fine-tuning, preference optimization, tool use, and distillation.
2 in the library.