Machine Translation

Why Does This Exist?

Language is not a 1-to-1 mapping. You cannot translate an English sentence to French by simply looking up every word in a dictionary and swapping it.

• Word order changes: English puts adjectives before nouns ("red car"), while French puts them after ("voiture rouge").
• Idioms: "It's raining cats and dogs" translates literally into nonsense in other languages.
• Context matters: "Bank" translates differently if the sentence is about a river versus money.

Machine Translation (MT) is the specific sub-field of NLP dedicated to solving this problem by learning the latent semantic meaning of a sentence and reconstructing it in a target language.

Think of It Like This

Think of It Like This

Imagine you are an interpreter at the United Nations listening to a Spanish diplomat.

You do not listen to a single Spanish word, immediately shout the English equivalent, and then wait for the next word. If you did, your translation would be a grammatical disaster.

Instead, you listen to an entire phrase. You build a mental picture of what the diplomat is trying to say (Encoding). Once you understand the idea, you construct a brand new English sentence to convey that idea (Decoding). Modern neural translation networks do exactly this.

How It Actually Works

Historically, translation relied on Rule-Based systems (hardcoded grammar rules) and Statistical Machine Translation (SMT), which relied on massive tables of word frequencies. Today, it is entirely solved by Neural Machine Translation (NMT).

The Seq2Seq Architecture

NMT uses a Sequence-to-Sequence (Seq2Seq) architecture, consisting of two parts:

1. The Encoder: Reads the source sentence in English. It compresses the entire sentence into a dense numerical vector (the "context vector") that represents the pure semantic meaning of the sentence.
2. The Decoder: Takes that context vector and generates the French translation one word at a time.

The Attention Mechanism

In early Seq2Seq models, the Encoder had to compress a 50-word sentence into a single vector, which caused it to "forget" the beginning of the sentence.

The Attention Mechanism fixed this. Instead of a single vector, the Encoder outputs a vector for every word in the source sentence. When the Decoder is trying to generate the French word for "apple", the Attention Mechanism allows it to dynamically "look back" at the exact English word "apple" in the source sentence. This mechanism was so successful it eventually led to the invention of the Transformer.

How We Measure Success

Translation is difficult to evaluate because there is rarely one single "correct" answer. "I am hungry" and "I have hunger" might both be valid translations depending on the language.

The industry standard metric is the BLEU (Bilingual Evaluation Understudy) Score. BLEU compares the machine's translation against one or more human reference translations. It counts the number of overlapping n-grams (phrases of 1, 2, 3, or 4 words) between the machine output and the human reference. A higher BLEU score (closer to 1.0 or 100) means the translation closely mimics human phrasing.

The Quick Version

• Machine Translation cannot be solved with word-for-word dictionary lookups due to grammar and idioms.
• Modern Neural Machine Translation uses an Encoder-Decoder (Seq2Seq) architecture.
• The Encoder builds a mathematical representation of the source sentence's meaning, and the Decoder uses it to generate the target language.
• The Attention Mechanism allows the Decoder to focus on specific parts of the source sentence while translating.
• We evaluate translation quality using the BLEU score, measuring overlap with human translations.

What to Read Next

• Transformer Architecture
• Text Summarization