ndif¶
Bases: dict
Status for remote execution on NDIF.
This class provides a structured view of the NDIF status, including information about all deployed models and their current states. It inherits from dict, allowing direct access to the underlying status data while providing rich formatting for display.
| ATTRIBUTE | DESCRIPTION |
|---|---|
status |
The overall service status (UP, REDEPLOYING, or DOWN).
TYPE:
|
Example
from nnsight import ndif_status status = ndif_status() print(status) # Displays a formatted table of all models status.status # Returns NdifStatus.Status.UP, etc.
| PARAMETER | DESCRIPTION |
|---|---|
response
|
Dictionary mapping repo_id to model info dictionaries, each containing 'model_class', 'repo_id', 'revision', 'type', and 'state'.
TYPE:
|
Bases: Enum
Overall NDIF service status.
| ATTRIBUTE | DESCRIPTION |
|---|---|
UP |
Service is operational with at least one model running.
|
REDEPLOYING |
Service is transitioning; models are deploying or starting.
|
DOWN |
Service is unavailable or no models are running.
|
Bases: Enum
Status of an individual model deployment.
| ATTRIBUTE | DESCRIPTION |
|---|---|
RUNNING |
Model is fully deployed and accepting requests.
|
DEPLOYING |
Model is currently being deployed.
|
NOT_DEPLOYED |
Model is configured but not yet started.
|
DOWN |
Model deployment has failed or is unavailable.
|
Bases: Enum
Type of model deployment on NDIF.
| ATTRIBUTE | DESCRIPTION |
|---|---|
DEDICATED |
Model is a permanent deployment.
|
PILOT_ONLY |
Model available only for pilot users.
|
SCHEDULED |
Model runs on a schedule (e.g., specific hours).
|
ndif_status(raw: bool = False) -> Union[dict, NdifStatus]
Query the current status of the NDIF service and all deployed models.
This is the primary function for checking NDIF availability and model status. When printed, the returned NdifStatus object displays a formatted table of all available models with their deployment type and current state.
| PARAMETER | DESCRIPTION |
|---|---|
raw
|
If True, return the raw API response dict. If False (default), return a formatted NdifStatus object.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Union[dict, NdifStatus]
|
If raw=True: The raw JSON response from the NDIF API. |
Union[dict, NdifStatus]
|
If raw=False: An NdifStatus object with formatted model information, or an empty dict if the request fails. |
Example
from nnsight import ndif_status status = ndif_status() print(status) NDIF Service: Up 🟢 ┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━┓ ┃ Model Class ┃ Repo ID ┃ Revision ┃ Type ┃ Status ┃ ┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━┩ │ LanguageModel │ meta-llama/Llama-3.1-70B │ main │ Dedicated │ RUNNING │ └───────────────┴────────────────────────────┴──────────┴───────────┴─────────┘
Checks if a specific model is currently running on NDIF.
| PARAMETER | DESCRIPTION |
|---|---|
repo_id
|
The HuggingFace repository ID (e.g., "meta-llama/Llama-3.1-70B").
TYPE:
|
revision
|
The model revision/branch to check. Defaults to "main".
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
bool
|
True if the model is running and available, False otherwise |
bool
|
(including if the API request fails). |
Example
from nnsight import is_model_running if is_model_running("meta-llama/Llama-3.1-70B"): ... print("Model is available!")