AI product Open source
DeepSelect is a high-performance CUDA and PyTorch implementation of TopK kernels for DeepSeek Sparse Attention and sampling workloads. It replaces vanilla `torch.topk` for supported inputs, covering bfloat16 lightning-indexer workloads and float32 sampling workloads, with configurable sorting, index type, value output, variable-length rows, and NaN handling. The repository reports 2–20× speedups against `torch.topk` on its benchmarks.
It is distributed as an installable Python package from the `deepseek-ai` GitHub repository. Supported TopK values are limited to 4096 or less, and input tensors must meet specified stride and contiguity requirements; the optimized sampling case targets vocabularies of approximately 128K.
2 uses taken from transcripts — each links to the moment in the video.
Replaces PyTorch's standard TopK operation with a custom CUDA kernel tuned for DeepSeek sparse attention and sampling. The project reports substantial speedups across supported indexing and sampling paths.
Provides tuned GPU kernels for top-k operations used in sparse-attention indexing and token sampling. Its tensor-in, tensor-out function is intended to improve inference throughput for these workloads.
2 in the library.