GPT-3 — language models are few-shot learners

The problem it solved

Fine-tuning a model per task needs labeled data and a training run for each one. Could a big enough model simply read a few examples in its prompt and generalize?

The key idea

Scale a decoder-only transformer to 175B parameters and pretrain on a huge corpus. At that scale, in-context learning emerges: the frozen model performs new tasks from a handful of prompt examples — zero-, one-, and few-shot.

What made it work

• Scale — parameters, data, and compute pushed well past prior models.
• In-context learning — the prompt itself becomes the "training signal", no weight updates.
• Task-agnostic pretraining — one model, many downstream uses.

Why it matters

GPT-3 reframed how we use LLMs: prompting replaced per-task fine-tuning for many applications, and "just make it bigger" became a serious research direction.