Lightning Indexer

Lightning Indexer is a lightweight learned selector that scores visible token or block candidates and returns the top-k indices used by core sparse attention.

Why It Exists

Sparse attention saves core compute only after deciding which parts of the context matter. Without a content-dependent selector, attention must either scan every token in the expensive main branch or rely on a fixed mask that can miss distant evidence. The Lightning Indexer supplies that routing decision for token-level DSA and compressed-block variants.

How It Works

For each query token, small indexer query and key projections score all visible candidates. DeepSeek-V3.2 DSA selects individual token positions; DeepSeek-V4’s compressed sparse attention scores compressed candidates and combines multiple indexer heads before top-k. Core attention then runs only over the selected positions. Because selection depends mainly on score ordering, serving frameworks can quantize the indexer cache and query to 8 bits—such as INT8 with per-token-head scales or FP8 on Ascend A5—while aiming to preserve the ranking.

Tradeoffs

The indexer is a heuristic: quantization or close score margins can reorder borderline candidates and change the top-k set. Its per-pair work is much smaller than main attention, but a full-prefix scan still has quadratic prefill complexity and linear-per-token decode cost. IndexCache reports that repeating this scan at every layer can dominate long-context prefill.

Common Confusions

Token-level vs. compressed-block selection: DeepSeek-V3.2 DSA selects token positions, while later compressed-attention designs select compressed entries or blocks. Both use a learned, query-dependent indexer rather than a fixed mask.
Indexer vs. MoE router: a router selects experts per token to reduce FFN compute; the indexer selects KV blocks per query to reduce attention compute.
C4 vs. C8: “C8” is the 8-bit quantization of the indexer cache and query implemented in vllm-ascend today; a 4-bit “C4” indexer quantization is not implemented yet (planned future work). Separately, “c4” is the DSV4 layer type (compress ratio 4) that owns an indexer.

Where It Appears

DeepSeek-V4 Inference on Ascend — Connects the indexer to the end-to-end DSA serving stack and heterogeneous KV cache.
DeepSeek-V4: Million-Token Context via Hybrid Compressed Attention — Introduces the Lightning Indexer as the sparse-selection stage of CSA layers.
DeepSeek-V4.1-Flash — Reuses indexer state across CSA2 layers and restricts later decoder indexers to a hierarchical candidate pool.
DeepSeek-V4 Lightning Indexer C8 Quantization — vllm-ascend’s INT8/FP8 indexer cache and quantized top-k operators.
MiniMax Sparse Attention — A sibling approach with a learned index branch for block selection.
IndexCache — Shows that adjacent DSA layers often produce redundant top-k selections and reuses the index tensor across layers.
DeepSeek-V3.2: Sparse Attention, Scaled RL, and Thinking in Tool-Use — DeepSeek-V3.2 introduces DeepSeek Sparse Attention (DSA) for sub-quadratic long-context efficiency, a scaled GRPO recipe with.
DeepSeek V4 Attention: Code Reading Map — A navigable map of the DeepSeek V4 hybrid compressed attention implementation across vLLM (NVIDIA/AMD/XPU) and vllm-ascend.
Triton in Practice: How vLLM and vllm-ascend Use Triton — A codebase-driven tour of Triton kernel development in vLLM (NVIDIA GPU) and vllm-ascend (Ascend NPU), covering infrastructure.
vLLM-Ascend Architecture: How the Ascend NPU Port Integrates with vLLM — A code-reading tour of how vllm-ascend maps onto vLLM’s six-layer stack and extends upstream vLLM for Ascend NPU execution.
vLLM Ascend — vLLM’s Ascend NPU port: code-reading notes and MoE forward implementation insights.
Qwen3.5 / Qwen3.6 Inference Path on vLLM Ascend — A code-reading tour of the shared qwen3_5-family inference path: Qwen3.5-27B / Qwen3.6-27B (dense hybrid Mamba-Transformer.
Efficient Attention Training — Training approaches built around efficient attention mechanisms: sparse, sliding-window, and delta-rule recurrent attention.
SWAT: Sliding Window Attention Training — Trains transformers from scratch with sigmoid-based sliding window attention, replacing softmax to eliminate attention sink and.

KV Cache — The attention state the indexer selects blocks from.
Cross-Layer Index Reuse — Reuses one indexer’s selected positions across several sparse-attention layers.
Mixture of Experts — A router-based sparse selection mechanism for FFN capacity, often confused with attention indexers.