Terms Glossary

Quick-lookup definitions for technical concepts that appear across multiple papers in this knowledge base. Each term page is a self-contained definition with a curated set of teaching examples; the generated term data supplies exhaustive consumer backlinks.

General

Agent Skill — A reusable, structured procedural artifact that guides an AI agent on a class of tasks without retraining the underlying model.
Monkey-Patching — Runtime replacement, deletion, or redirection of functions, attributes, modules, or environment lookups so code runs against controlled behavior without editing the original source.

Training

Pipeline Parallelism — Assigns layer ranges to stages and streams microbatches through them for model capacity and throughput.
Microbatch — A small chunk of a training batch used to enable pipeline parallelism; the unit of work injected into a pipeline schedule.
Mixture of Experts — Routes each token through a small subset of many expert feed-forward networks to increase total capacity without activating every parameter.
Hyper-Connections — Widens the residual stream into n parallel streams mixed by learnable mappings; mHC constrains that mixing to a doubly stochastic manifold to keep it trainable at scale.
Scatter/Gather — A pipeline-boundary optimization that sends different activation shards across slow links, then reconstructs the full tensor on the destination side with fast local collectives.
All-Gather — A many-to-many collective that gathers per-rank shards into one full tensor and delivers that complete result to every rank.
All-to-All — A many-to-many collective that scatters each rank’s data to all ranks while gathering from all ranks, effectively performing a distributed matrix transpose.
All-Reduce — A many-to-many collective that reduces equally shaped tensors across ranks and delivers the identical reduced result to every rank.
Sequence Parallelism — A distributed training strategy that splits the input sequence along the length dimension across GPUs, forming a fourth parallelism dimension orthogonal to data, pipeline, and tensor parallelism.
Tensor Parallelism — Splits weight matrices and hidden or head dimensions across accelerator ranks.

Algorithms

Context Parallelism — Distributes a long sequence and its KV state across ranks while preserving exact attention.
Chunked KV-Cache Compression — Summarizes consecutive token windows into fewer cached entries at a fixed stride, with phase-dependent retention.
Cross-Layer Index Reuse — Reuses dynamically selected token or block indices from an anchor sparse-attention layer in subsequent layers.
Gated Retention — Uses learned head-wise decay to carry a fixed-size recurrent key-value state across tokens.
Grouped-Query Attention — Shares each key/value head across a group of query heads to reduce KV-cache and communication cost.
Delta Rule — Corrects an associative memory using the error between its current key-addressed prediction and the target value.
General Matrix Multiply (GEMM) — The dense multiply-accumulate kernel C = A×B whose execution rate is the standard performance reference for linear-algebra workloads on GPUs and NPUs.
Inner Product — The scalar dot product Σ xᵢyᵢ; GEMM is the matrix of inner products between rows of A and columns of B, and attention scores are dot products.
Kimi Delta Attention — Extends delta-rule recurrent memory with channel-wise decay and hardware-efficient chunkwise computation.
Lightning Indexer — A lightweight learned selector that scores visible token or block candidates and returns the top-k indices used by core sparse attention.
Hadamard Transform — An orthogonal ±1 rotation that spreads a tensor’s outlier mass across all coordinates, enabling accurate low-bit LLM quantization.
Kronecker Product — The block-structured matrix product A⊗B that builds a large matrix from two smaller ones by scaling copies of B by the entries of A.
KV Cache — Stores earlier attention keys and values so autoregressive decoding reuses the prefix instead of recomputing it.
Layer Normalization — Standardizes each example or token across its feature dimensions, independently of other batch members.
Linear Attention — Factors query–key similarity through feature maps so key–value associations can be accumulated without an explicit quadratic attention matrix.
Low-Rank Approximation — Represents a matrix with fewer independent directions while preserving its dominant structure.
Looped Transformers — Reuse one block’s parameters across repeated applications to gain effective computational depth without adding parameters.
Matrix Tiling — Blocking a GEMM (or any kernel) into tiles that fit on-chip SRAM and registers so operands are loaded from global memory few times and reused many times.
Outer Product — A rank-1 matrix u vᵀ formed from two vectors; GEMM can be computed by accumulating outer products of columns of A with rows of B.
Perplexity — The exponential of a causal language model’s average next-token negative log likelihood on a specified tokenized corpus.
Ring Attention — Circulates query or KV blocks around a rank ring and merges partial softmax results into exact attention.
Sliding-Window Attention — Restricts causal attention to a fixed number of recent tokens so per-layer KV state stays bounded.

Hardware

Block Floating Point — A shared-scale numeric representation that amortizes exponent metadata and alignment work across narrow element payloads.
Chiplet — A package-level design that combines multiple smaller dies into one logical processor or memory system.
Collective Communication Unit (CCU) — A hardware engine that combines remote data movement and reductions for device-side collective communication.
FP8 — An 8-bit floating-point family used to reduce model and activation memory traffic at a controlled numerical cost.
Global Memory — The off-chip device memory on an accelerator (GPU HBM or Ascend GM) that holds full tensors; kernels move tiles from it into on-chip storage to compute.
GPTQ — A second-order post-training weight-quantization algorithm that compensates rounding errors and scales through shared column ordering and block updates.
HiF8 — Huawei’s 8-bit floating-point format with variable-width exponent encoding and a reported 38-power-of-two combined exponent range.
HiFloat4 (HiF4) — A 4.5-bit-per-value 64-element block-floating format with E6M2 base scaling, shared micro-exponents, and an integer-heavy dot-product path.
Memory Banking — Partitioning on-chip SRAM into banks so parallel accesses to different addresses hit different banks in the same cycle, avoiding bank conflicts.
Microscaling — A block-floating-point representation that shares one scale across a small group of narrow elements.
NDDMA — A multidimensional direct-memory-access engine that copies tensor data while applying layout transformations.
NVFP4 — NVIDIA’s four-bit floating-point format with E2M1 payloads, FP8 E4M3 micro-block scales, and a tensor-level scale in the public Blackwell recipe.
Post-Training Quantization — Applies quantization to an already-trained model using calibration data or weight statistics instead of full retraining.
SIMD/SIMT Hybrid Programming — A programming model that combines lane-level SIMD throughput with thread-level SIMT flexibility for regular and irregular vector work.
STARS Scheduler — Ascend’s device-side task and resource scheduler for coordinating compute, movement, synchronization, and accelerator engines.
Systolic Array — A regular grid of processing elements where data flows rhythmically between neighbors so each weight is reused across many multiply-accumulates without re-fetching.
Unified Bus (UB) — Huawei’s interconnect family for remote memory, asynchronous movement, collectives, and Ethernet-connected scale-out.
URMA — Unified Bus’s asynchronous remote-memory access semantic for copies, messages, and atomic operations.

Frameworks

Chunked Prefill — Splits a long prompt into causal chunks so prompt work can share iterations with decode work.
Continuous Batching — Rebuilds the active LLM-serving batch at each model iteration so finished work can leave and newly ready work can enter.
Block Table — The per-request logical-to-physical mapping that tells paged attention kernels which physical KV block holds each logical block of a sequence.
KV Cache Offloading — Moves request-specific key/value state to a slower memory or storage tier and fetches it during inference.
PagedAttention — An attention algorithm that stores a sequence’s KV cache in fixed-size non-contiguous blocks addressed through a per-request block table, enabling paged memory management.
Speculative Decoding — Drafts several tokens cheaply and has the target model verify them in parallel without changing the target distribution under strict acceptance.
Tree Attention — Evaluates speculative draft branches together while restricting every token to its own root-to-node ancestry.

Benchmarks

Benchmark Compression — Selects a small evaluation coreset that preserves the full benchmark’s scores, rankings, or other target statistics.