orkgnlp.common.service.runners.ORKGNLPTorchRunner

class ORKGNLPTorchRunner(*args)

Bases: ORKGNLPBaseRunner

The ORKGNLPTorchRunner is a runner specialized for Torch model formats. It requires therefore a model object of type torch.

Parameters:

model (Model object. See the inheriting classes for further information.) – The model to be run.

Methods

release_memory	Releases the memory of all available attributes in a pipeline component.
run	Runs the given model while initiation in evaluation mode and returns its output.

release_memory()

Releases the memory of all available attributes in a pipeline component.

run(inputs, multiple_batches=False, **kwargs)

Runs the given model while initiation in evaluation mode and returns its output.

Parameters:

• inputs (Union[Any, List[Tuple[Any]], Dict[str, Any], List[Dict[str, Any]]]) – Tuple of model arguments or dict of model named arguments. A list of tuples or a list of dicts in case of batches.

• multiple_batches (bool) – Whether the model is to be executed x times for each input instance or batch, where x is the length of inputs list. Note that in this case the model’s outputs will be returned as a python generator. Defaults to False.

Return type:

Union[Tuple[Any, Dict[str, Any]], Tuple[Generator[Any, None, None], Dict[str, Any]]]

Returns:

The model output as a tuple or list of tuples, and kwargs.