AI product Open source
MiniMax Sparse Attention (MSA) is an MIT-licensed Python and CUDA kernel package from MiniMax for dense FlashAttention and block-sparse top-k attention on NVIDIA SM100 GPUs. Its sparse prefill path uses a dense proxy pass to compute per-block maximum scores, selects the highest-scoring key-value blocks with a top-k indexer, and runs attention only over those blocks; the repository also provides paged FP8 decode and support for BF16, FP8, NVFP4, and FP4 paths through its CuTe-DSL stack. The package contains two runtime-compiled implementations: Jinja-generated CUDA kernels and a CuTe-DSL implementation, connected by an adapter for replacing dense-API calls with sparse attention. It requires Linux x86_64, Python 3.10 or later, CUDA with nvcc, and an NVIDIA SM100 GPU.
1 use taken from transcripts — each links to the moment in the video.
MiniMax Sparse Attention, the sparse attention architecture used to train M3: an index branch that selects important token blocks and a sparse branch that attends only to those, giving about 9x faster prefill and 15x faster decoding than full attention.
1 in the library.