Skip to content
AI360Xpert
Gen AI

Activation Steering

Instead of fine-tuning a model to change its behavior, what if we just find the 'politeness' vector in its brain and mathematically inject it into every prompt?

Activation steering alters a model's behavior by mathematically injecting a specific concept vector directly into its hidden states during inference.
Activation steering alters a model's behavior by mathematically injecting a specific concept vector directly into its hidden states during inference.

Why Does This Exist?

Historically, if you wanted an LLM to behave a certain way (e.g., to act like a Pirate, to be more polite, or to refuse toxic prompts), you had two options:

  1. Prompt Engineering: Tell it to be polite in the system prompt. (Easily circumvented by jailbreaks).
  2. Fine-tuning / RLHF: Train the model on thousands of examples of polite text. (Extremely expensive, computationally heavy, and mathematically permanent).

Mechanistic Interpretability gave us a third option. Through techniques like probing-classifiers, we learned that high-level concepts (like "Politeness", "Deception", or "Sycophancy") are actually stored as specific mathematical vectors inside the model's hidden layers.

Activation Steering (also called Representation Engineering or Control Vectors) asks a radical question: If a concept is just a vector, can we just grab that vector, and mathematically add it to the model's brain while it is running?

The answer is yes. You can temporarily "brainwash" a model without training it at all.

Think of It Like This

Think of It Like This

Think of fine-tuning an LLM like sending someone to acting school for a year so they can learn how to play a Pirate in a movie. It takes a long time, and once they learn it, it permanently changes how they act.

Activation Steering is like a sci-fi brain implant. You press a button on a remote control, and the "Pirate" module activates in their brain. They instantly talk like a pirate. When you turn the remote off, they instantly revert to normal. It is immediate, temporary, and requires zero training.

How It Actually Works

Steering a model requires two distinct phases: finding the vector, and injecting the vector.

1. Finding the Control Vector

First, you need to isolate the concept you want. Let's say you want a "Pirate" vector.

  1. You generate 500 normal prompts ("Hello", "How to cook eggs", "Write a poem").
  2. You generate 500 pirate prompts ("Ahoy matey", "How to plunder a ship", "Write a sea shanty").
  3. You run all 1,000 prompts through the LLM. You extract the hidden state vectors from the middle layers (e.g., Layer 15) for every prompt.
  4. You take the mathematical average of all the Pirate vectors, and subtract the mathematical average of all the Normal vectors.

The resulting difference is the Pirate Control Vector. You have successfully isolated the exact mathematical representation of "Pirateness" in this specific model.

2. Injecting the Vector

Now, a user logs into your application and types a completely normal prompt: "Please give me a recipe for pancakes."

As the data flows through the LLM, you pause it at Layer 15. You take the user's hidden state, and you mathematically add the Pirate Control Vector to it:

New_Hidden_State = Original_Hidden_State + (Pirate_Vector * Steering_Multiplier)

You then let the model continue processing Layer 16 to the end. The output will magically read: "Arrr, ye want to make pancakes, do ye? First, pillage some flour from the galley..."

Show Me the Code

Injecting vectors requires a library that can intercept the forward pass of a PyTorch model. Libraries like TransformerLens or Hugging Face's accelerate hooks are commonly used.

import torch
def steer_forward_pass(    model,     input_ids,     steering_vector,     target_layer_idx=15,     steering_strength=1.5):    """    Injects a control vector into a specific layer during inference.    """        # Define a hook function that will trigger at the target layer    def steering_hook(module, input, output):        # The 'output' is the hidden state vector of this layer        # We simply ADD our steering vector to it!        steered_output = output[0] + (steering_vector * steering_strength)                # Return the modified vector back to the model        return (steered_output,) + output[1:]        # 1. Attach the hook to the target layer    target_layer = model.transformer.h[target_layer_idx]    hook_handle = target_layer.register_forward_hook(steering_hook)        # 2. Run the forward pass!     # The model will be "steered" automatically as it passes layer 15.    with torch.no_grad():        outputs = model(input_ids)            # 3. Clean up the hook so the model returns to normal    hook_handle.remove()        return outputs

Watch Out For

The Goldilocks Coefficient

When injecting a vector, you must set a steering strength (multiplier). If you set it too low (0.1), the model ignores it. If you set it too high (5.0), you completely destroy the model's linguistic capabilities, and it will output unreadable gibberish (e.g., "Arrr arrr arrr arrr"). Finding the exact multiplier that alters behavior without destroying grammar requires extensive trial and error.

The Quick Version

  • Activation Steering (Representation Engineering) allows you to change an LLM's behavior without fine-tuning it.
  • You first isolate a concept (like "Politeness") by extracting hidden states from polite prompts and subtracting normal prompts.
  • During inference, you intercept the hidden state in the middle of the network and mathematically add this "Politeness" vector.
  • The model immediately adopts the behavior for that specific forward pass.
  • This technique is actively being researched for AI Alignment, as you can inject a "Refusal" or "Honesty" vector to force a model to reject jailbreaks or stop hallucinating.
  • sparse-autoencoders — The ultimate tool for mechanistic interpretability, which allows us to find millions of these semantic vectors automatically.

Related concepts