orkgnlp.common.service.runners.ORKGNLPONNXRunner

class ORKGNLPONNXRunner(*args)[source]

Bases: ORKGNLPBaseRunner

The ORKGNLPONNXRunner is a runner specialized for ONNX model formats. It requires therefore a model object of type onnx.

Parameters

model (Model object. See the inheriting classes for further information.) – The model to be run.

Methods

release_memory

Releases the memory of all available attributes in a pipeline component.

run

Runs the given model while initiation in evaluation mode and returns its output.
release_memory()

Releases the memory of all available attributes in a pipeline component.

run(inputs, output_names=None, custom_input_dict=None, **kwargs)[source]

Runs the given model while initiation in evaluation mode and returns its output.

Parameters

  • inputs (Tuple[Any]) – Tuple of model arguments.

  • output_names (Optional[List[str]]) – List of output names of the ONNX graph. Check your exporting code for further information! Defaults to None.

  • custom_input_dict (Dict[str, List[Any]].) – When given, the argument inputs will be ignored. This argument must have the following schema: {input_name_0: [input_value_0], …, input_name_n: [input_value_n]}. Check your exporting code for further information! Defaults to None.

Return type

Tuple[Any, Dict[str, Any]]

Returns

The model output and kwargs.