Info
Last Execution: 2026-09-02
| Package | Version |
|---|---|
| nnsight | 0.8.0 |
| Python | 3.12.13 |
| torch | 2.13.0+cu126 |
| transformers | 5.15.0.dev0 |
Setting Activations¶
Setting is how you intervene on a model by editing activations as they flow through the network. This is the basis of techniques like activation patching, ablation, and steering.
There are two ways to set a value:
- In-place (
module.output[:] = new) mutates the tensor the model is already holding, so every later read of that value sees the change. - Replacement (
module.output = new) hands the model a brand new tensor to continue with, leaving the original object untouched.
Setup¶
from nnsight import TransformersModel
model = TransformersModel("openai-community/gpt2", device_map="auto", dispatch=True)
In-Place Setting¶
Use slice assignment to modify a tensor's values in-place. The original tensor object is mutated, so downstream modules see the change immediately.
What .output looks like depends on the model and the transformers version — a module may hand back a plain tensor or a tuple, so always verify with print(module), type(...), or .shape before indexing. For this GPT-2 build, model.transformer.h[0].output is a (batch, seq, hidden) tensor, so we index it directly — there is no [0] to unwrap.
with model.trace("The Eiffel Tower is in the city of"):
# Clone before the edit so we can compare
before = model.transformer.h[0].output.clone().save()
# Zero out all activations at layer 0
model.transformer.h[0].output[:] = 0
after = model.transformer.h[0].output.save()
print("Before:", before[0, 0, :5])
print("After: ", after[0, 0, :5])
Before: tensor([ 0.1559, -0.7946, 0.3943, 0.3413, -0.5653], device='cuda:0',
grad_fn=<SliceBackward0>)
After: tensor([0., 0., 0., 0., 0.], device='cuda:0', grad_fn=<SliceBackward0>)
Clone before saving in-place modifications
When modifying in-place, the saved reference and the modified tensor point to the same memory. If you want to capture the "before" state, call .clone() before the modification.
Replacement Setting¶
Assign a completely new tensor to a module's output. This schedules a swap that replaces the value the model uses downstream, rather than mutating the existing tensor. No need to .clone() when comparing before/after.
with model.trace("The Eiffel Tower is in the city of"):
before = model.transformer.wte.output.save()
# Replace the embedding output with a scaled version
model.transformer.wte.output = model.transformer.wte.output * 0.5
after = model.transformer.wte.output.save()
print("Before:", before[0, 0, :5])
print("After: ", after[0, 0, :5])
Before: tensor([-0.0686, -0.0203, 0.0645, -0.0621, -0.1135], device='cuda:0',
grad_fn=<SliceBackward0>)
After: tensor([-0.0343, -0.0101, 0.0322, -0.0310, -0.0568], device='cuda:0',
grad_fn=<SliceBackward0>)
Setting Specific Positions¶
You can target specific batch items, token positions, or hidden dimensions using standard indexing.
with model.trace("The Eiffel Tower is in the city of"):
# Zero out only the last token position at layer 5
model.transformer.h[5].output[:, -1, :] = 0
output = model.lm_head.output.save()
predicted = model.tokenizer.decode(output[0, -1].argmax(dim=-1))
print(f"Predicted next token (after zeroing last position at layer 5): {predicted}")
Predicted next token (after zeroing last position at layer 5): ,
Adding a Steering Vector¶
A common intervention is adding a direction vector to activations to steer model behavior.
import torch
# A random direction is still random — seed it so this cell is reproducible.
torch.manual_seed(0)
with model.trace("The Eiffel Tower is in the city of"):
hidden = model.transformer.h[6].output
# Create a random steering vector on the same device
steering = torch.randn(hidden.shape[-1], device=hidden.device) * 10
# Add it to the last token position
model.transformer.h[6].output[:, -1, :] += steering
output = model.lm_head.output.save()
predicted = model.tokenizer.decode(output[0, -1].argmax(dim=-1))
print(f"Predicted next token (after steering): {predicted}")
Predicted next token (after steering): the
Handling Tuple Outputs¶
Whether a module's .output is a plain tensor or a tuple varies by model and transformers version, so check with print(module) / type(...) before you index. In this GPT-2 build the transformer blocks hand back a plain tensor while some submodules return tuples: the attention module here returns (attn_out, ...), so its tensor lives at .output[0]. Use slice assignment on that element to modify it in-place, or rebuild the tuple to replace it.
with model.trace("The Eiffel Tower is in the city of"):
# In-place: modify the first element of the tuple
model.transformer.h[0].attn.output[0][:] = 0
output = model.transformer.h[0].attn.output.save()
print(f"Attention output is a {type(output).__name__} with {len(output)} elements")
Attention output is a tuple with 2 elements
Tuple assignment
model.layer.output[0][:] = 0 modifies the tensor inside the tuple in-place. Assigning to a tuple element directly — model.layer.output[0] = new_tensor — raises TypeError, since tuples don't support item assignment. To replace an element, rebuild and reassign the whole tuple: model.layer.output = (new_tensor,) + tuple(model.layer.output[1:]).
Verifying Downstream Effects¶
Setting an early layer's output affects all downstream computations. Here we compare the final logits with and without an intervention.
with model.trace("The Eiffel Tower is in the city of"):
clean_logits = model.lm_head.output.save()
with model.trace("The Eiffel Tower is in the city of"):
model.transformer.h[0].output[:] = 0
modified_logits = model.lm_head.output.save()
diff = (clean_logits - modified_logits).abs().mean()
print(f"Mean absolute difference in logits: {diff:.4f}")
print(f"Clean prediction: {model.tokenizer.decode(clean_logits[0, -1].argmax(dim=-1))}")
print(f"Modified prediction: {model.tokenizer.decode(modified_logits[0, -1].argmax(dim=-1))}")
Mean absolute difference in logits: 59.8508 Clean prediction: Paris Modified prediction: ,