Core ML

Sparse Autoencoders for Features

Neurons inside an LLM are polysemantic—they fire for many unrelated things at once. A sparse autoencoder takes those messy neuron activations and disentangles them into a massive dictionary where each entry represents exactly one clear, human-readable concept.

A dense, messy hidden state is expanded into a massive dictionary of features, forced by an L1 penalty to only activate a tiny handful of them at any given time.

Why Does This Exist?

If you want to understand how a Large Language Model works, the obvious first step is to look at its neurons. If you find a neuron that fires when the model reads French text, you found the "French neuron."

But when researchers actually looked, they hit a wall: Superposition (superposition-and-circuits). Because the model has to learn millions of concepts but only has a few thousand neurons per layer, it compresses them. A single neuron might fire for French text, baseball statistics, and HTML tags. It is polysemantic. You cannot read the network by looking at individual neurons.

To understand the network, we need to untangle that compressed knot back into individual, single-meaning features. Sparse Autoencoders (SAEs) are the tool that finally cracked this open. They act as a translator, taking a messy, dense neural state and splitting it into a clean, readable dictionary of concepts.

Think of It Like This

Imagine a crowded room where 50 different conversations are happening at once.

If you record the room with just two microphones (the dense hidden state), the audio track is a complete mess. Every frequency contains pieces of ten different voices.

A Sparse Autoencoder acts like an AI audio separator. It takes that dense, tangled stereo track and outputs 50 separate audio channels (the dictionary of features). The trick that makes it work is sparsity: it assumes that at any exact millisecond, most people in the room are quiet. By forcing most of the 50 channels to be silent at any given moment, it successfully isolates the individual voices.

How It Actually Works

An SAE is a separate, much simpler neural network trained on the activations of the LLM.

1. The Setup

You run billions of tokens through the LLM and collect the hidden states from one specific layer. If the LLM has a hidden dimension of $d = 4,096$ , your dataset is billions of vectors of size 4,096.

2. The Autoencoder

You build an autoencoder with a single hidden layer. But unlike a standard autoencoder that bottlenecks the data to compress it, an SAE expands it.

The hidden layer (the "dictionary") is much wider than the input—often $16 \times$ or $64 \times$ wider. If the input is 4,096 dimensions, the dictionary might have 131,072 features.

3. The L1 Penalty (The Secret Sauce)

If you just train a wide autoencoder to reconstruct the input, it learns nothing useful. The magic is the L1 regularization penalty applied to the dictionary layer.

The L1 penalty charges the model a steep cost for every feature it turns on. To minimize the loss, the SAE is forced to turn on the absolute minimum number of features necessary to reconstruct the dense input. Out of 131,072 available features, only 10 or 20 might be active for any given token.

4. The Result

Because the SAE is forced to be sparse, it can't afford to smear concepts across multiple features. It assigns one clear concept to one feature.

When researchers inspect the trained dictionary, they find features that are perfectly monosemantic. There is a feature that only fires for base64 strings, another that only fires for the concept of deception, and another that only fires for apologies.

Watch Out For

The hallucination of features. An SAE is an approximation. It tries to reconstruct the LLM's hidden state, but it is never 100% perfect. Sometimes, the SAE will output a feature that looks incredibly interpretable—say, a "sarcasm" feature—but if you manually activate that feature (using activation-patching) to see if it forces the LLM to write sarcastically, nothing happens.

Always remember that the SAE is a lens you placed over the model, not the model itself. To prove a feature is real, you must causally patch it back into the LLM and measure the behavioral change.

The Quick Version

The Problem: LLM neurons are polysemantic. They represent multiple unrelated concepts simultaneously due to superposition.
The Fix: Train a Sparse Autoencoder to expand the dense hidden state into a massive dictionary of features.
The Engine: An L1 penalty forces the SAE to activate only a tiny handful of features at a time, forcing it to disentangle the concepts.
The Result: We get human-readable, single-meaning features (like a "Golden Retriever" feature or a "coding error" feature) that we can map and manipulate.