Knowledge note
vLLM Ascend
vLLM's Ascend NPU port: code-reading notes and MoE forward implementation insights.
vLLM Ascend
The pinned vLLM Ascend source record and code reading notes at commit 51b0242c0d3be1d3bd59e763a7d78b8553cd45e1 retain the October 2026 checkpoint analysis evidence.
- Qwen3.5 MTP: Drafting and Target-Model Verification — Concrete Qwen3.5 MTP path: target hidden state plus token embedding create draft proposals, target logits are aligned to each proposed position, and vLLM’s rejection sampler accepts a prefix, recovers the first rejection, or appends a bonus token.
- vLLM-Ascend Architecture: How the Ascend NPU Port Integrates with vLLM — Mapped onto vLLM’s six-layer stack (reuse layers 1-4, replace layers 5-6), then the five integration mechanisms (plugin registration, NPUPlatform, ModelRegistry, monkey-patches, custom backends), worker variants (NPUWorker/NPUWorker310/XliteWorker), execution flow from startup to per-step inference, ACL graph capture, HCCL communication, and what upstream vLLM code is reused as-is.
- GLM-5.2 on vLLM Ascend: Request-to-Backend Inference Path — Ascend NPU companion to the upstream GPU trace: request admission and scheduling through the
glm_moe_dsamodel shell, SFA shared-indexer path, paged sparse attention, routed/shared MoE, sampling, response emission, and cleanup. - GLM-5.1 W4A4C8 on Ascend 950: How vLLM-Ascend Turns On Sparse C8 — How the independent SFA-C8 and Lightning-Indexer-C8 switches build packed versus key/scale caches, select A5 FP8/FP32 cache dtypes, and reach the fused Ascend 950 operators.
- vLLM-Ascend Kimi K3 MoE Forward Insight — Latest-code insight for the Kimi K3-style routed-MoE forward substrate in vllm-ascend: patched FusedMoE construction, typed MoE stage contracts, Ascend routing, token dispatch, grouped MLP compute, routed-expert capture, Fused MC2, and dynamic EPLB.
- DeepSeek-V4 Lightning Indexer C8 Quantization — How the DeepSeek-V4 Lightning Indexer runs on an 8-bit key cache and query in vllm-ascend: INT8 + FP16 scales on 910B/A2/A3, FP8 e4m3 + FP32 scales on A5, the quantized top-k custom operators, and the C4-vs-C8 naming.
- DeepSeek-V4 Inference on Ascend: The DSA Serving Stack — End-to-end DeepSeek-V4 serving on Ascend NPUs: the model override with mHC hyper-connections, hybrid c4/c128 compressor layers, the AscendDSA prefill/decode flow, the five-type heterogeneous KV cache, the sparse-attention custom operator, and the MTP draft model.
- Qwen3.5 / Qwen3.6 Inference Path on vLLM Ascend — How the shared
qwen3_5-family models (Qwen3.5-27B / Qwen3.6-27B dense hybrid Mamba-Transformer, multimodal; Qwen3.5/Qwen3.6-35B-A3B and Qwen3.5-397B-A17B sparse MoE) run on Ascend by reusing theqwen3_5/qwen3_5_moemodel types: patched hybrid decoder forward, GDN linear attention plus FIA full attention, ModelSlim W8A8 quantization,qwen3_5_mtpspeculative decoding, and ACL-graph capture at pinned revision9a52ca5fc36c. - Qwen3.8-FP8 on Ascend 950: Why Weights Become MXFP8 at Load Time — How native Qwen/Qwen3.8-27B-FP8 block-wise FP8 weights are reconstructed, re-quantized to E4M3 + E8M0 MXFP8 on Ascend 950, and repacked for
npu_quant_matmul, with dense fallbacks on other Ascend generations. - vLLM-Ascend Prefill and Decode Scheduling: Qwen3.5 GQA — Token-budget scheduling, chunked prefills, mixed FIA GQA layouts, and the parallel GDN recurrent path.
Related: vLLM — the upstream NVIDIA/AMD/XPU serving framework.