Multi-Query Attention
An optimized transformer mechanism where all attention heads share a single set of Key and Value projections while maintaining multiple distinct Queries.
Think of It Like This
Like a dozen different customers asking unique questions to a single librarian who shares the exact same reference book with all of them.
MQA was designed to aggressively reduce the memory bandwidth bottleneck during autoregressive decoding. By storing only one KV pair per token instead of one per head, the KV cache size drops dramatically. While it significantly speeds up inference, it can slightly degrade model quality compared to full multi-head attention.