Knowledge note
Attention Algorithms
Topic hub for exact, approximate, and architecture-specific attention mechanisms.
Attention Algorithms
- FlashAttention — IO-aware exact-attention kernel family.
- Attention Variants — Multi-query, grouped-query, collaborative, and latent attention.
- Linear Attention — Kernelized and recurrent alternatives to quadratic attention.
- Phase Sensitivity in Chunked KV-Cache Compression — Shows stride-periodic retrieval gaps and phase-specialized contributions in compressed models.