Getting Started#
NNsight (/ɛn.saɪt/) is a package for the interpreting and manipulating the internals of deep learning models.
Installation#
To get started with NNsight, install it with pip.
pip install nnsight
Please give the project a star on Github to support the project. NNsight is open-source and you can read and clone the full source at ndif-team/nnsight.
Remote Model Access#
To remotely access LLMs through NDIF, you must sign up for an NDIF API key.
NDIF hosts multiple LLMs, including various sizes of the Llama 3.1 models and DeepSeek-R1 models. All of our models are open for public use, but you need to apply for access to the Llama-3.1-405B models. You can view the full list of hosted models at https://nnsight.net/status/.
If you have a clear research need for Llama-3.1-405B and would like more details about applying for access, please refer to our 405B pilot program application.
Access LLM Internals#
Now that you have your NDIF API key, you can start exploring LLM internals with NDIF and NNsight. We’ve put together a Colab notebook to help you get started.
This notebook will walk you through the following steps:
Installing NNsight
Setting up your NDIF API key
Loading a LLM in NNsight
Accessing and altering LLM internals remotely
Next Steps#
Walk through the basic functionality of the package.
Configure API access for remote model execution.
Check out the basic features provided by nnsight.
See nnsight implementations of common interpretability techniques.
Discuss nnsight, NDIF, and more!