Logit Lens
An interpretability technique that decodes the intermediate hidden states of a transformer model directly into vocabulary words to observe its 'thinking'.
Think of It Like This
Like putting a stethoscope on a person's brain to hear their half-formed thoughts before they actually speak a full sentence.
By applying the model's final unembedding matrix to internal layers, researchers can see how a model's prediction evolves as data passes through the network. This reveals where the model resolves facts, changes its mind, or retrieves specific knowledge. It is a vital tool for mechanistic interpretability.