Tutorials¶
Comprehensive tutorials covering core concepts and research techniques.
Getting Started¶
- Walkthrough - Complete introduction to nnsight
- Chat Templates - Working with chat-formatted models
- Remote Access - Getting started with NDIF
Migrating¶
- From TransformerLens - Port a TransformerLens script: the hook-name/module-path table, and the weight-processing conventions that make a correct port produce different numbers
Causal Tracing¶
- Causal Models Intro - Introduction to causal modeling
- Activation Patching - Patching activations to trace information flow
- Attribution Patching - Gradient-based attribution methods
- Causal Mediation I - Causal mediation analysis part 1
- Causal Mediation II - Causal mediation analysis part 2
- DAS - Distributed Alignment Search
Probing¶
- Logit Lens - Interpreting intermediate representations
- Diffusion Lens - Probing diffusion models
- Concept Erasure - Removing a linear subspace inside the forward pass, and measuring what it costs
Attention¶
- Attention Patterns and Per-Head Detection - Reading the attention probability matrix with
.source, and scoring every head for induction, previous-token and sink behaviour
Circuits¶
- Attribution Graphs - Replace the MLPs with a cross-layer transcoder and trace which features caused an output
Architectures¶
- MoE Routing - Reading a mixture-of-experts router, and the two ablations that quietly measure the wrong thing
- State-Space Models - Reading and writing the recurrent state of a Mamba-style model, where there is no KV cache
Engineering¶
- Dataset-Scale Activation Caching - Running a corpus to disk: throughput, memory, resumability, and the anchor that says the cache still matches the model
Steering¶
- Evaluating SAEs - Given a pretrained sparse autoencoder, is it any good?
- Dictionary Learning - Learning interpretable feature dictionaries
- LoRA Tutorial - Low-rank adaptation with nnsight
Vision¶
- Cross-Attention Ablation - Localising where a prompt attaches in Stable Diffusion