backend¶
Run a trace on a remote nnsight-serve engine.
The backend for model.trace(..., serve=url). The trace is written against a
meta model — no GPU, no weights — serialized in the same form the remote (NDIF)
path uses, and sent to a running nnsight-serve instance. The server runs it and
returns the saved values, which are pushed back into the caller's frame exactly as
a local run would, so reading a .save()d value after the block just works::
model = VLLM("gpt2") # no GPU needed
with model.trace("Hello", serve="http://host:8000", api_key="..."):
logits = model.logits.save()
logits # pushed back here
api_key (optional) is sent as the ndif-api-key header. The server's response
is a torch.save of {"saves": {name: value}, "error": <deferred or None>} —
saved values only, so read generated tokens by saving tracer.result. A build or
runtime error comes back as error and is re-raised at the client with its real type and traceback; a
transport/service failure (e.g. the engine not ready) surfaces as ConnectionError.