AI product Open source

DeepSelect

DeepSelect is a high-performance CUDA and PyTorch implementation of TopK kernels for DeepSeek Sparse Attention and sampling workloads. It replaces vanilla `torch.topk` for supported inputs, covering bfloat16 lightning-indexer workloads and float32 sampling workloads, with configurable sorting, index type, value output, variable-length rows, and NaN handling. The repository reports 2–20× speedups against `torch.topk` on its benchmarks.

View repository Mentioned in 2 videos ↓

Overview

It is distributed as an installable Python package from the `deepseek-ai` GitHub repository. Supported TopK values are limited to 4096 or less, and input tensors must meet specified stride and contiguity requirements; the optimized sampling case targets vocabularies of approximately 128K.

What DeepSelect is used for

2 uses taken from transcripts — each links to the moment in the video.

  • Replaces PyTorch's standard TopK operation with a custom CUDA kernel tuned for DeepSeek sparse attention and sampling. The project reports substantial speedups across supported indexing and sampling paths.

  • Provides tuned GPU kernels for top-k operations used in sparse-attention indexing and token sampling. Its tensor-in, tensor-out function is intended to improve inference throughput for these workloads.

Videos mentioning DeepSelect

2 in the library.