iterator¶
Targeting specific occurrences of a location within one run.
A traced call can reach the same module more than once — most commonly a
generation loop, where every module is revisited on each decoded step. Each
visit produces a fresh occurrence of that module's locations (its
.input/.output/...). Iterations, reached as tracer.iter,
lets a stretch of trace body bind its reads and writes to a chosen range of
those occurrences:
.. code-block:: python
with model.generate("...", max_new_tokens=10) as tracer:
for step in tracer.iter[:3]:
hidden = model.transformer.h[0].output.save() # steps 0, 1, 2
The mechanism lives in the interleaver (see
nnsight.intervention.interleaver): each visit to a location is tagged
with its occurrence index, and a worker's request is tagged with the occurrence
it wants — the running Mediator's
iteration. Looping over tracer.iter is what moves that pointer.
Bases: Tracer
Selects which occurrences of a location a stretch of trace body targets.
Returned by iter. A
module reached repeatedly in one run — once per step of a generation loop,
say — produces a fresh occurrence of each of its locations every step.
Looping over tracer.iter walks the running
Mediator's iteration pointer
across a range of those occurrences, so reads and writes inside the loop body
bind to the matching step:
.. code-block:: python
with model.generate("...", max_new_tokens=10) as tracer:
for step in tracer.iter[:3]:
hidden = model.transformer.h[0].output.save() # steps 0, 1, 2
The occurrences are chosen by indexing:
- a slice —
tracer.iter[:3]is steps 0–2,tracer.iter[2:5]is 2–4; - an int —
tracer.iter[2]is just step 2; - a list —
tracer.iter[[0, 2, 4]]is those steps only (the skipped occurrences still advance the model's per-location count, so index 2 binds the third occurrence regardless of which steps were selected).
An open end — tracer.iter[:] — runs until the model stops producing
steps: the loop keeps handing out step indices, and the final request (for a
step the model never runs, e.g. one past the last generated token) is simply
left parked. The interleaver reports that dangling request as a warning rather
than an error (see
check_dangling_mediators),
keeping the values from every step that did run.
An end the user named — tracer.iter[:10], iter[2], iter[[0, 2, 4]]
— that the run cannot supply ends the same way: cut short, with the same
warning. The loop is the user's own for statement and the worker is
parked inside its body, so the only way out is to unwind it — values saved
inside the loop are kept, and whatever the block does after the loop does
not run. The warning says so, and names what pins a generation to the
loop's count (min_new_tokens= on transformers, min_tokens= /
ignore_eos=True on vLLM) for when the shortfall was not meant.
A with tracer.iter[...]: block does the same thing and is deprecated; the
loop moves inside execute, which re-runs the block per step. Prefer the
for form.
| ATTRIBUTE | DESCRIPTION |
|---|---|
start |
First occurrence index for a range; ignored when
|
stop |
One past the last for a range, or
|
steps |
An explicit list of occurrence indices, or
|
__getitem__(key: slice | int | list[int]) -> Iterations
Select occurrences via [:n] / [a:b] / [i] / [[i, j, ...]].
Walk the running mediator's iteration across the selected steps.
Before yielding each step, pin the mediator's iteration so the first
request in the loop body binds to that occurrence (it then relaxes; see
iteration). An open end
(stop is None) keeps handing out steps indefinitely — the model
stopping is what ends the loop, via a dangling final request. Whatever
iteration was before the loop is restored on exit, so loops can nest.
Run the block once per selected occurrence — the deprecated with form.
with tracer.iter[...]: predates for step in tracer.iter[...]: and
does the same thing the long way: rather than the loop being the user's,
it is here, re-running the captured block with the mediator's iteration
pinned to each step (see __iter__ for the same pinning). Prefer the
for form, which is a plain loop over the body written inline.
Open-ended (tracer.iter[:]) ends the way the for form does: the
step past the model's last parks, and the interleaver throws
OutOfOrderError into it once the
run finishes — caught here, so the reached steps' saved values are kept.