ts.torch_handler package

Subpackages

• ts.torch_handler.request_envelope package
  • Submodules
  • ts.torch_handler.request_envelope.base module
  • ts.torch_handler.request_envelope.body module
  • ts.torch_handler.request_envelope.json module
  • ts.torch_handler.request_envelope.kserve module
  • ts.torch_handler.request_envelope.kservev2 module
  • Module contents
• ts.torch_handler.unit_tests package
  • Subpackages
    • ts.torch_handler.unit_tests.models package
      • Submodules
      • ts.torch_handler.unit_tests.models.base_model module
      • Module contents
    • ts.torch_handler.unit_tests.test_utils package
      • Submodules
      • ts.torch_handler.unit_tests.test_utils.mock_context module
      • Module contents
  • Submodules
  • ts.torch_handler.unit_tests.test_base_handler module
  • ts.torch_handler.unit_tests.test_envelopes module
  • ts.torch_handler.unit_tests.test_image_classifier module
  • ts.torch_handler.unit_tests.test_image_segmenter module
  • ts.torch_handler.unit_tests.test_mnist_kf module
  • ts.torch_handler.unit_tests.test_object_detector module
  • Module contents

Submodules

ts.torch_handler.base_handler module

Base default handler to load torchscript or eager mode [state_dict] models Also, provides handle method per torch serve custom model specification

class ts.torch_handler.base_handler.BaseHandler

Bases: abc.ABC

Base default handler to load torchscript or eager mode [state_dict] models Also, provides handle method per torch serve custom model specification

describe_handle()

Customized describe handler

Returns

A dictionary response.

Return type

dict

explain_handle(data_preprocess, raw_data)

Captum explanations handler

Parameters

• data_preprocess (Torch Tensor) – Preprocessed data to be used for captum

• raw_data (list) – The unprocessed data to get target from the request

Returns

A dictionary response with the explanations response.

Return type

dict

handle(data, context)

Entry point for default handler. It takes the data from the input request and returns

the predicted outcome for the input.

Parameters

• data (list) – The input data that needs to be made a prediction request on.

• context (Context) – It is a JSON Object containing information pertaining to the model artefacts parameters.

Returns

Returns a list of dictionary with the predicted response.

Return type

list

inference(data, *args, **kwargs)

The Inference Function is used to make a prediction call on the given input request. The user needs to override the inference function to customize it.

Parameters

• data (Torch Tensor) – A Torch Tensor is passed to make the Inference Request.

• shape. (The shape should match the model input) –

Returns

The Predicted Torch Tensor is returned in this function.

Return type

Torch Tensor

initialize(context)

Initialize function loads the model.pt file and initialized the model object.

First try to load torchscript else load eager mode state_dict based model.

Parameters

• context (context) – It is a JSON Object containing information

• parameters. (pertaining to the model artifacts) –

Raises

RuntimeError – Raises the Runtime error when the model.py is missing

postprocess(data)

The post process function makes use of the output from the inference and converts into a Torchserve supported response output.

Parameters

data (Torch Tensor) – The torch tensor received from the prediction output of the model.

Returns

The post process function returns a list of the predicted output.

Return type

List

preprocess(data)

Preprocess function to convert the request input to a tensor(Torchserve supported format). The user needs to override to customize the pre-processing

Args :

data (list): List of the data from the request input.

Returns

Returns the tensor data of the input

Return type

tensor

ts.torch_handler.contractions module

contraction map for text classification models.

ts.torch_handler.densenet_handler module

Module for image classification default handler

class ts.torch_handler.densenet_handler.DenseNetHandler

Bases: object

DenseNetHandler handler class. This handler takes an image and returns the name of object in that image.

handle(data, context)

Entry point for default handler

inference(data, *args, **kwargs)

Override to customize the inference :param data: Torch tensor, matching the model input shape :return: Prediction output as Torch tensor

initialize(context)

First try to load torchscript else load eager mode state_dict based model

ts.torch_handler.densenet_handler.list_classes_from_module(module, parent_class=None)

Parse user defined module to get all model service classes in it.

Parameters

• module –

• parent_class –

Returns

List of model service class definitions

ts.torch_handler.image_classifier module

ts.torch_handler.image_segmenter module

ts.torch_handler.object_detector module

ts.torch_handler.text_classifier module

ts.torch_handler.text_handler module

ts.torch_handler.vision_handler module

Module contents