orkgnlp.common.service.runners.ORKGNLPTorchRunner
- class ORKGNLPTorchRunner(*args)[source]
Bases:
ORKGNLPBaseRunner
The ORKGNLPTorchRunner is a runner specialized for Torch model formats. It requires therefore a model object of type
torch
.- Parameters
model (Model object. See the inheriting classes for further information.) – The model to be run.
Methods
Releases the memory of all available attributes in a pipeline component.
Runs the given model while initiation in evaluation mode and returns its output.
- release_memory()
Releases the memory of all available attributes in a pipeline component.
- run(inputs, multiple_batches=False, **kwargs)[source]
Runs the given model while initiation in evaluation mode and returns its output.
- Parameters
inputs (
Union
[Any
,List
[Tuple
[Any
]],Dict
[str
,Any
],List
[Dict
[str
,Any
]]]) – Tuple of model arguments or dict of model named arguments. A list of tuples or a list of dicts in case of batches.multiple_batches (
bool
) – Whether the model is to be executed x times for each input instance or batch, where x is the length ofinputs
list. Note that in this case the model’s outputs will be returned as a python generator. Defaults to False.
- Return type
Union
[Tuple
[Any
,Dict
[str
,Any
]],Tuple
[Generator
[Any
,None
,None
],Dict
[str
,Any
]]]- Returns
The model output as a tuple or list of tuples, and kwargs.