Skip to content

Tutorials

Comprehensive tutorials covering core concepts and research techniques.

Getting Started

Migrating

  • From TransformerLens - Port a TransformerLens script: the hook-name/module-path table, and the weight-processing conventions that make a correct port produce different numbers

Causal Tracing

Probing

  • Logit Lens - Interpreting intermediate representations
  • Diffusion Lens - Probing diffusion models
  • Concept Erasure - Removing a linear subspace inside the forward pass, and measuring what it costs

Attention

Circuits

  • Attribution Graphs - Replace the MLPs with a cross-layer transcoder and trace which features caused an output

Architectures

  • MoE Routing - Reading a mixture-of-experts router, and the two ablations that quietly measure the wrong thing
  • State-Space Models - Reading and writing the recurrent state of a Mamba-style model, where there is no KV cache

Engineering

Steering

Vision