Mechanistic Interpretability
A research field dedicated to reverse-engineering neural networks down to individual neurons and circuits to understand exactly how they compute answers.
Think of It Like This
Like taking apart a complex digital watch piece by piece to understand precisely how the gears turn to produce the correct time.
Instead of treating models as black boxes, this field seeks to map high-level behaviors to low-level computational subgraphs. Techniques include activation patching and sparse autoencoders to isolate concepts. The ultimate goal is to guarantee AI safety by mathematically proving how models derive their outputs.