AI product Open source
Miles is an enterprise-facing reinforcement learning framework for large-scale post-training of large language and vision-language models. It pairs SGLang for high-throughput, agentic rollouts with Megatron-LM for scalable training, and also provides a PyTorch FSDP2 backend for Hugging Face implementations. Its asynchronous architecture decouples rollout and training workers, supports configurable on- and off-policy schedules, and updates rollout engines in-loop through peer-to-peer RDMA weight transfer. The framework includes token-in-token-out data flow, Rollout Routing Replay for replaying mixture-of-experts routing decisions during training, fault-tolerant recovery of failed SGLang engines, low-precision training with MXFP8 and NVFP4 alongside FP8, INT4 QAT, BF16, and FP16, and LoRA or multi-LoRA training. It supports reinforcement-learning recipes including GRPO, GSPO, PPO, and REINFORCE++, as well as supervised fine-tuning, on-policy distillation, agentic environments, and diffusion-model training. Miles was forked from slime and integrates SGLang, Megatron-LM, and torch_memory_saver; the repository is released as an open-source project, though the provided page text does not state its license.
1 use taken from transcripts — each links to the moment in the video.
An open-source enterprise reinforcement-learning framework for post-training language and vision-language models. It supports asynchronous rollout and training, multiple GPU platforms, training methods such as GRPO and SFT, and connectors for coding and computer-use agents.
1 in the library.