From NLP to Transformers
Trace the path that led from bag-of-words to attention, so the transformer feels like an answer, not a mystery.
Who this is for
Learners who want the story behind transformers — why each idea appeared and what problem it solved.
How to use this path
Read it as history. Each step exists because the previous approach hit a wall: embeddings fixed sparse one-hot vectors, RNNs added order, attention fixed the fixed-length bottleneck, and transformers removed recurrence for parallelism.
What you'll be able to do
Motivate every part of the transformer from first principles — and explain, in an interview, why attention replaced recurrence.