Inference Serving Systems

• vLLM — GPU serving architecture, scheduling, caches, deployment, and model paths.
• vLLM Ascend — Ascend NPU integration and model-specific execution paths.
• SGLang — Structured language-model programming and serving.
• Sarathi — Chunked-prefill scheduling for efficient inference.
• KV Cache Optimization — Eviction, compression, hybrid-memory, and alternative-attention strategies.
• KVSwap — Disk-aware KV-cache offloading for on-device inference.
• InferenceX — Continuous inference benchmark and execution pipeline.
• DeepSeek Integrations — Framework-level DeepSeek implementation readings.
• AgentX / InferenceX v3 — Agentic inference workload and hardware comparison.