tracer

tracer

A tracer whose workers can be built without running the model.

The base InterleavingTracer builds a trace's workers and, in the same step, runs the forward. An async vLLM trace instead hands the request to the engine and streams the outputs, so it needs the worker-building step on its own, before any forward. VLLMTracer factors that step into prepare; the synchronous path (execute) still runs it and then the forward, exactly as the base does. Keeping this in the vLLM package leaves the base tracer untouched.

VLLMTracer

VLLMTracer(envoy: Envoy, fn: Callable | str, *args: Any, backend: Backend | None = None, **kwargs: Any)

Bases: InterleavingTracer

An InterleavingTracer whose worker-building is callable on its own.

barrier

barrier(n: int) -> None

Not available here — see no_barrier.

prepare

prepare(code: CodeType) -> tuple

Build the trace's workers and combined call input, without running the model.

The first half of execute: collect the invoke workers (or the single direct-input worker) onto the interleaver and assemble the batched call input, then return the workers alongside it. The async backend uses this to get the workers to serialize and the input to submit, in place of interleave.

RETURNS	DESCRIPTION
tuple	(workers, args, kwargs) — the workers to read results from, and the
tuple	combined (args, kwargs) for the model call.

execute

execute(code: CodeType) -> None

Build the workers, run the forward interleaved, push results back.

no_barrier

no_barrier(n: int) -> None

Refuse a barrier, which this runtime cannot hold.

A barrier releases when n of a trace's blocks have reached it, which needs them all running against one forward. Here each invoke is a separate vLLM request, scheduled independently and possibly in different steps entirely, so the blocks never coexist and the barrier would simply never release — a hang rather than an error. Say so at the call instead.