Getting Started#
NNsight (/ɛn.saɪt/) is a package for the interpreting and manipulating the internals of deep learning models.
Installation#
To get started with NNsight, install it with pip.
pip install nnsight
Please give the project a star on Github to support the project. NNsight is open-source and you can read and clone the full source at ndif-team/nnsight.
Remote Model Access#
To remotely access LLMs through NDIF, you must sign up for an NDIF API key.
NDIF hosts multiple LLMs, including various sizes of the Llama 3.1 models and DeepSeek-R1 models. All of our models are open for public use, but you need to apply for access to the Llama-3.1-405B models. You can view the full list of hosted models at https://nnsight.net/status/.
If you have a clear research need for Llama-3.1-405B and would like more details about applying for access, please refer to our 405B pilot program application.
Access LLM Internals#
Now that you have your NDIF API key, you can start exploring LLM internals with NDIF and NNsight. We’ve put together a Colab notebook to help you get started.
This notebook will walk you through the following steps:
Installing NNsight
Setting up your NDIF API key
Loading a LLM in NNsight
Accessing and altering LLM internals remotely
Next Steps#
Walk through the basic functionality of the package.
Configure API access for remote model execution.
Check out the basic features provided by nnsight.
See nnsight implementations of common interpretability techniques.