batching¶ batching ¶ VLLMBatcher ¶ VLLMBatcher(*args, **kwargs) Bases: Batcher current_module instance-attribute ¶ current_module = None parallel instance-attribute ¶ parallel = False gathered instance-attribute ¶ gathered = False type instance-attribute ¶ type = None wrap ¶ wrap(model: Envoy) check_gathered ¶ check_gathered() narrow ¶ narrow(batch_group: Union[int, None]) swap ¶ swap(batch_group: Union[int, None], swap_value: Any)