Wiki Index

Start Here

Algorithms

Algorithms — Category overview.
Foundations — Transformer, normalization, probability, sequence, and linear algebra.
Attention Algorithms — Exact, approximate, architecture-specific, and compressed-KV attention.
Distributed Inference — Multi-device inference algorithms.
Length Generalization — Extrapolating computation beyond training lengths.
Model Architecture Algorithms — Model-wide algorithmic reports.

Benchmarks

Benchmarks — Category overview.
Agent Evaluation — Coding, tool-use, interaction, and skill-learning benchmarks.
Serving Performance — Inference stress testing and performance roadmaps.

Frameworks

Frameworks — Category overview.
Inference Serving Systems — Runtimes, caches, and deployment pipelines.
Compilers and Kernel Frameworks — GPU and NPU programming stacks.
Programming Languages — C++ and Python semantics and runtimes.
Runtime Infrastructure — Sandboxing, orchestration, and observability.
Speculative Decoding Systems — Draft-model and diffusion-based acceleration.

Hardware and Numerics

Hardware and Numerics — Category overview.
Accelerator Architectures — NVIDIA GPU and Ascend NPU families.
Numerics and Quantization — Low-precision formats and quantization.
Hardware Kernels and Dataflow — Communication-aware kernels and spatial execution.

Training

Training — Category overview.
Training Parallelism — Tensor, pipeline, and sequence parallelism.
Foundation Models — GPT and LLaMA papers.
Model Families — DeepSeek and Kimi reports.
Efficient Attention Training — Sparse, recurrent, and shared-cache attention.
Model Adaptation — Fine-tuning, transfer, and agent self-evolution.
Training Architectures — Training-motivated architectural mechanisms.