Features

Getting

Access values

notebooks/features/getting.ipynb

Setting

Intervene on values

notebooks/features/setting.ipynb

Scan and Validate

Debug tensor shapes

notebooks/features/scan_validate.ipynb

Modules

Apply modules

notebooks/features/modules.ipynb

Gradients

Intervene on gradients

notebooks/features/gradients.ipynb

Early Stopping

Save computation time

notebooks/features/early_stopping.ipynb

Skip

Skip module execution

notebooks/features/skip.ipynb

Source

Access and intervene on intermediate values

notebooks/features/source.ipynb

Cross Prompts

Edit in one pass

notebooks/features/cross_prompt.ipynb

Generation

Generate multiple tokens

notebooks/features/multiple_token.ipynb

Model Editing

Add persistent interventions

notebooks/features/model_editing.ipynb

Remote Execution

Use our servers

notebooks/features/remote_execution.ipynb

Streaming

Send remote values to local

notebooks/features/streaming.ipynb

vLLM Support

Fast inference

notebooks/features/vllm_support.ipynb

Report Issues

NNsight and NDIF are open-source and you can report issues, read, and clone the full source at ndif-team/nnsight. Also check out https://discuss.ndif.us/ to ask questions about our features or suggest new ones.

• Cross-Prompt Intervention and Batching
• Early Stopping
• Getting
• Gradients
• Model Editing
• Modules
• Multiple Token Generation
• Remote Execution
• Scan and Validate
• Setting
• Order Matters
• Skipping Modules
• Source Code Interventions
• Streaming
• vLLM Support