Shortcuts

ts.torch_handler package

Subpackages

Submodules

ts.torch_handler.base_handler module

Base default handler to load torchscript or eager mode [state_dict] models Also, provides handle method per torch serve custom model specification

class ts.torch_handler.base_handler.BaseHandler[source]

Bases: abc.ABC

Base default handler to load torchscript or eager mode [state_dict] models Also, provides handle method per torch serve custom model specification

describe_handle()[source]

Customized describe handler

Returns

A dictionary response.

Return type

dict

explain_handle(data_preprocess, raw_data)[source]

Captum explanations handler

Parameters
  • data_preprocess (Torch Tensor) – Preprocessed data to be used for captum

  • raw_data (list) – The unprocessed data to get target from the request

Returns

A dictionary response with the explanations response.

Return type

dict

handle(data, context)[source]
Entry point for default handler. It takes the data from the input request and returns

the predicted outcome for the input.

Parameters
  • data (list) – The input data that needs to be made a prediction request on.

  • context (Context) – It is a JSON Object containing information pertaining to the model artefacts parameters.

Returns

Returns a list of dictionary with the predicted response.

Return type

list

inference(data, *args, **kwargs)[source]

The Inference Function is used to make a prediction call on the given input request. The user needs to override the inference function to customize it.

Parameters
  • data (Torch Tensor) – A Torch Tensor is passed to make the Inference Request.

  • shape should match the model input shape. (The) –

Returns

The Predicted Torch Tensor is returned in this function.

Return type

Torch Tensor

initialize(context)[source]
Initialize function loads the model.pt file and initialized the model object.

First try to load torchscript else load eager mode state_dict based model.

Parameters
  • context (context) – It is a JSON Object containing information

  • to the model artifacts parameters. (pertaining) –

Raises

RuntimeError – Raises the Runtime error when the model.py is missing

postprocess(data)[source]

The post process function makes use of the output from the inference and converts into a Torchserve supported response output.

Parameters

data (Torch Tensor) – The torch tensor received from the prediction output of the model.

Returns

The post process function returns a list of the predicted output.

Return type

List

preprocess(data)[source]

Preprocess function to convert the request input to a tensor(Torchserve supported format). The user needs to override to customize the pre-processing

Args :

data (list): List of the data from the request input.

Returns

Returns the tensor data of the input

Return type

tensor

ts.torch_handler.contractions module

contraction map for text classification models.

ts.torch_handler.densenet_handler module

Module for image classification default handler

class ts.torch_handler.densenet_handler.DenseNetHandler[source]

Bases: object

DenseNetHandler handler class. This handler takes an image and returns the name of object in that image.

handle(data, context)[source]

Entry point for default handler

inference(data, *args, **kwargs)[source]

Override to customize the inference :param data: Torch tensor, matching the model input shape :return: Prediction output as Torch tensor

initialize(context)[source]

First try to load torchscript else load eager mode state_dict based model

ts.torch_handler.densenet_handler.list_classes_from_module(module, parent_class=None)[source]

Parse user defined module to get all model service classes in it.

Parameters
  • module

  • parent_class

Returns

List of model service class definitions

ts.torch_handler.image_classifier module

ts.torch_handler.image_segmenter module

ts.torch_handler.object_detector module

ts.torch_handler.text_classifier module

ts.torch_handler.text_handler module

ts.torch_handler.vision_handler module

Module contents

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources