ndif

ndif

Top-level NDIF helpers.

• register — ship a local module with a remote request (serialize by value).
• status / is_model_running — query the NDIF service and its models.
• compare — diff the local Python environment against the remote one.

Tables render as plain text with optional ANSI color (matching nnsight.intervention.backends.display); no rich dependency. Network libraries are imported lazily so importing nnsight stays light.

CRITICAL_PACKAGES module-attribute

CRITICAL_PACKAGES = {'nnsight', 'transformers', 'torch'}

NdifStatus

NdifStatus(deployments: dict)

A view of NDIF's deployed models, with a formatted table on print.

deployments maps repo id -> {model_class, repo_id, task, trust_remote_code, revision, level, state, hot_replicas} for each HOT/WARM model; status is the derived service state. Indexing/iteration delegate to deployments for convenience. Replicas of one deployment fold into one entry (healthiest state wins; hot_replicas counts the HOT ones). A checkpoint deployed under several tasks appears once per task, keyed "repo_id (task)" — with the revision joining the key when the tasks tie, and trust_remote_code when those tie too.

deployments instance-attribute

deployments = deployments

status instance-attribute

status = self._derive_status()

Status

Bases: str, Enum

UP class-attribute instance-attribute

UP = 'UP'

REDEPLOYING class-attribute instance-attribute

REDEPLOYING = 'REDEPLOYING'

DOWN class-attribute instance-attribute

DOWN = 'DOWN'

__getitem__

__getitem__(key: str) -> dict

__iter__

__iter__()

__contains__

__contains__(key: str) -> bool

__len__

__len__() -> int

keys

keys()

__str__

__str__() -> str

__repr__

__repr__() -> str

EnvComparison

EnvComparison(local_env: dict, remote_env: dict)

A local-vs-remote environment diff, inspectable and printable.

Printing shows the Python-version line and the package table. Inspect the structured result directly:

• local / remote — the raw env dicts.
• local_python / remote_python / python_matches.
• packages — {pkg: {"local", "remote", "match", "critical"}} for every package the server has.
• mismatches / critical_mismatches — the differing subset.

local instance-attribute

local = local_env

remote instance-attribute

remote = remote_env

local_python instance-attribute

local_python = local_env.get('python_version', 'Unknown').split()[0]

remote_python instance-attribute

remote_python = remote_env.get('python_version', 'Unknown').split()[0]

packages instance-attribute

packages = {pkg: {'local': local_pkgs.get(pkg, '-'), 'remote': remote_pkgs[pkg], 'match': local_pkgs.get(pkg, '-') == remote_pkgs[pkg], 'critical': pkg.lower() in CRITICAL_PACKAGES} for pkg in remote_pkgs}

python_matches property

python_matches: bool

mismatches property

mismatches: dict

critical_mismatches property

critical_mismatches: dict

__str__

__str__() -> str

__repr__

__repr__() -> str

register

register(module: Any) -> None

Register a local module for serialization by value in remote execution.

Code submitted to NDIF that imports a module not installed on the server would raise ModuleNotFoundError. Registering the module ships its class and function source with the request so it's rebuilt server-side. Thin wrapper over cloudpickle.register_pickle_by_value (the serializer nnsight uses honors it).

PARAMETER	DESCRIPTION
module	The module object, or its name as a string. TYPE: Any

Example::

import mymodule
import nnsight
nnsight.register(mymodule)           # or nnsight.register("mymodule")

with model.trace("Hello", remote=True):
    out = mymodule.myfn(model).save()

status

status(raw: bool = False) -> Union[dict, NdifStatus]

Query the NDIF service and its deployed models.

Printing the returned NdifStatus shows a table of the currently deployed (HOT/WARM) models and their state.

PARAMETER	DESCRIPTION
raw	If True, return the raw /status JSON instead of an NdifStatus. TYPE: bool DEFAULT: False

RETURNS	DESCRIPTION
Union[dict, NdifStatus]	The raw dict (raw=True), else an NdifStatus (empty, with a
Union[dict, NdifStatus]	DOWN status, if the service is unreachable).

Examples:

>>> import nnsight
>>> print(nnsight.status())          # table of deployed models
>>> "openai-community/gpt2" in nnsight.status()

ndif_status

ndif_status(raw: bool = False) -> Union[dict, NdifStatus]

Deprecated: use status.

is_model_running

is_model_running(repo_id: str, revision: str = 'main', task: Optional[str] = None, trust_remote_code: Optional[bool] = None) -> bool

Whether repo_id (at revision) is currently RUNNING on NDIF.

Returns False if the service is unreachable. The repo id is canonicalized via the Hub so different spellings match the deployed key. task and trust_remote_code narrow the check to one deployment, since both are part of the model key; unset, any value counts.

Examples:

>>> import nnsight
>>> nnsight.is_model_running("openai-community/gpt2")
>>> nnsight.is_model_running("openai-community/gpt2", task="text-generation")
>>> nnsight.is_model_running("x/custom", trust_remote_code=True)

whoami

whoami(api_key: Optional[str] = None) -> dict

Resolve the identity NDIF associates with an API key.

Calls the service's /whoami endpoint and returns {"email": ..., "tags": [...]}. email is None when the key is unrecognized (or the server has key validation disabled). Uses the configured key (CONFIG.API.APIKEY) when api_key isn't given.

PARAMETER	DESCRIPTION
api_key	The key to resolve; defaults to the configured one. TYPE: Optional[str] DEFAULT: None

RETURNS	DESCRIPTION
dict	The /whoami identity dict, {"email": str | None, "tags": list}.

login

login(api_key: Optional[str] = None) -> None

Store your NDIF API key so future sessions can use it (HuggingFace-style).

Prompts for the key with getpass (never echoed) when not given, verifies it against the service via whoami, then persists it with CONFIG.set_default_api_key (which writes config.yaml). Verification is best-effort — the key is still saved if the service can't be reached or doesn't recognize it, with a note — so a typo is surfaced without blocking login.

PARAMETER	DESCRIPTION
api_key	The NDIF API key. Prompted for (hidden) when not given; empty input is a no-op that saves nothing. TYPE: Optional[str] DEFAULT: None

Examples:

>>> from nnsight import login
>>> login()
Enter your NDIF API key:
NDIF API key saved — logged in as you@example.com.

main

main() -> None

Console entry point for the nnsight command (login / whoami).

get_local_env

get_local_env() -> dict

The local Python version and installed packages (by import name).

get_remote_env

get_remote_env(force_refresh: bool = False) -> dict

The NDIF server's Python version and installed packages (cached).

pull_env

pull_env() -> None

Auto-register local (non-installed) modules for serialize-by-value, once.

The remote backend calls this before its first request: every module get_local_env discovers as "local" (importable from the working tree, not a pip install) is passed to register, so its source ships with remote requests. Without it, remote code that imports a local module raises ModuleNotFoundError server-side. Cached via _PULLED_ENV so the (cheap but non-trivial) local-env scan runs only once per process.

build_table

build_table(local_env: dict, remote_env: dict) -> str

A package version-comparison table (local vs remote) as plain text.

compare

compare() -> EnvComparison

Compare the local and remote NDIF Python environments.

Package or Python-version drift between client and server can make interventions behave differently remotely than locally; this surfaces it.

Returns an EnvComparison — print it for the table, or inspect .mismatches / .critical_mismatches / .python_matches.

Examples:

>>> import nnsight
>>> print(nnsight.compare())         # local vs remote package table
>>> nnsight.compare().critical_mismatches