Gen AI

Circuits and Superposition

LLMs don't just memorize data; they build tiny, logical sub-networks called 'circuits' that execute specific reasoning algorithms (like 'find the previous name and repeat it').

In Mechanistic Interpretability, individual features link together to form circuits: small, logical sub-networks that execute specific reasoning algorithms.

Why Does This Exist?

Through tools like sparse-autoencoders, researchers solved the problem of Superposition (the fact that LLMs compress millions of concepts into thousands of neurons). By untangling the network, we can now point to a specific feature and say, "This feature means 'Dog'."

But finding isolated features is only step one. A dictionary of words doesn't explain how a book was written. To truly understand how an AI "thinks," we need to understand how it combines these isolated features to execute logic.

Circuits are the holy grail of Mechanistic Interpretability. A circuit is a small, understandable sub-network inside the massive LLM that executes a specific, human-readable algorithm. By finding and documenting these circuits, researchers are slowly reverse-engineering the black box of AI, proving that LLMs don't just rely on statistical vibes—they actually compute logic.

Think of It Like This

Think of an LLM like a giant, messy warehouse full of loose electronic parts (Superposition).

A Feature (found by a Sparse Autoencoder) is like identifying a single battery, a switch, or a lightbulb. It is a fundamental building block.

A Circuit is discovering that someone wired the battery to the switch, and the switch to the lightbulb. You have discovered a functional algorithm. If you flip the switch, the light turns on. By documenting these circuits, you stop viewing the warehouse as a pile of junk, and start seeing it as a structured machine.

How It Actually Works

Finding a circuit requires intense, manual detective work, often tracing the path of data backward from the final prediction using specialized tools.

1. The Indirect Object Identification (IOI) Task

One of the most famous circuits ever discovered in an LLM is the IOI circuit. It solves sentences like: "John and Mary went to the store, then Mary gave a drink to..."

Any human knows the next word is "John." But how does a math equation figure that out?

2. Tracing the Circuit

Researchers at Anthropic and Redwood Research traced the math backward and found a highly reliable circuit consisting of three main parts:

Name Mover Heads: Specific attention heads late in the network whose entire job is to copy names from earlier in the sentence and paste them at the end.
S-Inhibition Heads: Middle-layer attention heads whose job is to aggressively suppress (inhibit) the name of the subject who is currently acting. Because "Mary" is the one giving the drink, the S-Inhibition head fires a negative mathematical signal against the "Mary" token.
The Result: The Name Mover Heads look back at the sentence, see "John" (normal signal) and "Mary" (heavily negative signal). They mathematically select "John" and copy it to the output.

3. The Power of Circuits

The researchers proved this circuit exists by mathematically disabling the S-Inhibition heads. As soon as they turned those heads off, the model started predicting "Mary gave a drink to Mary"—proving that the circuit was the exact mechanism driving the reasoning.

The Most Famous Circuit: Induction Heads

Perhaps the most important circuit ever discovered is the Induction Head. It is a two-layer circuit whose only job is to execute the following logic: "If I see token [A] followed by token [B] earlier in the text, and I see token [A] again now, I should predict token [B]."

This simple circuit is the mathematical engine behind in-context learning (few-shot prompting). When you give an LLM a pattern like "France -> Paris, Japan -> Tokyo, Italy -> ?", the Induction Heads look back, recognize the [Country] -> [Capital] pattern, and output "Rome". Researchers proved that models cannot do few-shot learning until Induction Heads mathematically form during training.

Show Me the Code

You cannot easily "code" a circuit discovery tool in a few lines. It requires massive libraries like TransformerLens to cache and intervene on the internal activations of a model. However, here is the conceptual workflow of how you prove a circuit exists using "Activation Patching."

1# Conceptual Workflow for Proving the IOI Circuit2
3# 1. Run a clean prompt and get the correct output ("John")4clean_prompt = "John and Mary went... Mary gave a drink to"5clean_activations = get_all_internal_math(model, clean_prompt)6# Output: John (90%)7
8# 2. Run a corrupted prompt where the answer should be different9corrupted_prompt = "John and Peter went... Peter gave a drink to"10corrupted_activations = get_all_internal_math(model, corrupted_prompt)11# Output: John (90%)12
13# 3. Activation Patching14# We take the mathematical output of the "S-Inhibition Head" from the 15# CLEAN run, and surgically inject it into the CORRUPTED run.16patched_model = model.patch(17    target_node="s_inhibition_head_layer_8",18    new_data=clean_activations["s_inhibition_head_layer_8"]19)20
21# 4. Measure the result22patched_output = patched_model(corrupted_prompt)23
24# If the output magically changes from "John" to "Mary", you have proven25# that the S-Inhibition Head is the exact circuit component responsible 26# for solving the puzzle!

Watch Out For

The Illusion of Completeness

Circuits are incredibly satisfying to find, but they are often just the tip of the iceberg. A model might use the primary IOI circuit 90% of the time, but if you disable it, the model might fall back on a dozen backup circuits that are much harder to interpret. Assuming one clean circuit explains 100% of a model's behavior is a dangerous oversimplification in Mechanistic Interpretability.

The Quick Version

Superposition explains how LLMs pack millions of concepts into thousands of neurons. Sparse Autoencoders help untangle those concepts into readable "Features."
Features are just nouns. "Circuits" are the verbs—the actual algorithms that connect features together to solve logic puzzles.
By tracing data backward through the network, researchers have found circuits that handle things like pronoun resolution, pattern matching (Induction Heads), and factual recall.
Proving a circuit exists allows us to treat the AI not as a statistical black box, but as a readable, debuggable software program.

Related concepts

Attention is Not Explanation