Multi-Head Attention
A core mechanism in transformers that runs multiple attention calculations in parallel, allowing the model to simultaneously focus on different features.
Think of It Like This
Like reading a complex legal document while one lawyer checks for loopholes, another for grammar, and a third for context.
Instead of computing a single set of attention scores, the input is linearly projected into multiple smaller 'heads.' Each head learns to capture different linguistic or semantic relationships (e.g., subject-verb agreement vs. sentiment). The outputs are concatenated, providing a rich, multi-faceted understanding of the sequence.