GPUWorker¶

GPUWorker ¶

NNsightGPUWorker ¶

NNsightGPUWorker(*args, **kwargs)

Bases: Worker

Custom vLLM GPU worker that uses :class:NNsightGPUModelRunner.

Monkey-patches the default GPUModelRunner class before initialization so vLLM creates NNsight-aware model runners that can execute intervention code during model forward passes.

init_device ¶

init_device()

collect_nnsight ¶

collect_nnsight(req_ids: list[str], finished_req_ids: list[str] | None = None)